Low latency Logging (BrightonPHP - 18th Nov 2013)
2 likes3,985 views
The document presents a talk on low latency logging, emphasizing the importance of logging for error handling in PHP applications. It discusses various methods and technologies such as RabbitMQ for enhancing logging efficiency, alongside PSR-3 standards and practical logging implementations. The importance of structured error handling and capturing mechanisms is highlighted to improve application reliability.
Low latency Logging (BrightonPHP - 18th Nov 2013)
- 1. Low Latency Logging at BrightonPHP
- 3. ● Started on Amiga 500 ● PHP for 11 years ● ZCE PHP 5.3 ● ● ● ● ● Run PHP Hampshire user group Lead dev on browscap.ini First open source GoDeploy Development Manager at Protected.co.uk Mostly a ZF2 developer @asgrim github.com/asgrim
- 5. Errors
- 6. What are errors? ● Something broke… :) ● e.g. ○ Can’t connect to MySQL (mysqli_connect) ○ No such file or directory (fopen) ● Usually PHP core ● Sometimes “fatal”
- 8. Problems. ● ● ● ● ● error_reporting setting Errors look ugly “@” Limited options Not very “OO”
- 9. Ways Around // Will raise E_NOTICE fopen($somefile, 'r');
- 10. Ways Around // No error! :) if (file_exists($somefile)) { fopen($somefile, 'r'); } else { // nice handling... // maybe throw exception... }
- 11. Exceptions
- 12. Jargon Buster ● throw Triggering ● try Try to run ● catch Handle it ● finally Run code after try/catch
- 13. What are exceptions? ● ● ● ● ● Something still broke OO Catchable Fatal errors They are classes
- 14. SPL Exceptions ● Built in to PHP ● More descriptive than just “Exception”, e.g.: ○ ○ ○ ○ ○ InvalidArgumentException LogicException OutOfBoundsException RuntimeException see PHP manual for more
- 15. Example (exception class) class DivisionByZeroException extends LogicException { }
- 16. Example (throw) class Mathematics { public function divide($a, $b) { if ($b == 0) { $msg = sprintf(‘Tried to divide %s by zero”, $a); throw new DivisionByZeroException($msg); } return ($a / $b); } }
- 17. Example (catch) $maths = new Mathematics(); try { $result = $maths->divide(5, 0); } catch (DivisionByZeroException $exception) { $logger->warning($exception>getMessage()); }
- 18. Logging
- 20. What is “logging”? Keeping a record of all events... exceptions, errors, warnings, info, debug
- 21. Why use logging? ● ● ● ● Easier to find problems More detail “paper trail” for code Log where you want
- 22. What about Apache’s error_log? source: http://up-ship.com/blog/?p=20903
- 23. Why? ● error_log is too basic ● Reading / parsing ● error_reporting (again)
- 24. Doin’ it right wrong… /************************************************************ * Magic file that makes your entire project work perfectly * ************************************************************/ @ini_set('display_errors', 0); @error_reporting(0); function __globalErrorHandler() { return true; } @set_error_handler('__globalErrorHandler'); @set_exception_handler('__globalErrorHandler'); @register_shutdown_function(function() { if(error_get_last()) { echo "Script executed successfully!"; } }); https://github.com/webarto/boostrap.php
- 25. Requirements (for everyone) ● ● ● ● Fire & forget Minimum or zero latency Highly available Log everything: ○ ○ ○ ○ Exceptions Errors Fatal Errors Debug & info ● PSR-3 compatible
- 26. PSR-3 ● Common logging interface → LoggerInterface ● RFC-5424 Levels (debug, info, notice, warning, error, critical, alert, emergency) ● Interoperability ● Reusability
- 27. Logging Options ● ● ● ● ● ● ● monolog (PSR-3) Drupal - PSR-3 Watchdog phpconsole log4php RavenPHP + Sentry FirePHP (dev environment) Roll your own
- 28. How they work... source: http://mirror-sg-wv1.gallery.hd.org/_c/natural-science/cogs-with-meshed-teeth-AJHD.jpg.html
- 29. Capturing Logging Use “capture methods”, send to $logger ● set_exception_handler() ○ Handles all uncaught exceptions ● set_error_handler() ○ Handles most errors ● register_shutdown_function() ○ Handles fatal errors
- 30. Sending Log Messages ● Handler/Adapter translates ● However you want… ● Monolog has loads: ○ ○ ○ ○ syslog-compatible / error_log Email, HipChat AMQP, Sentry, Zend Monitor, Graylog2 Redis, MongoDB, CouchDB
- 31. Typical PSR-3 Compatible Design Capture Method Logger (PSR-3) Handler / Adapter Data Storage
- 34. What do I mean? ● Easy to add ● Fire & Forget ● Minimum impact to request
- 35. Slow Logging Browser Application Log Server HTTP request Error! Send log message to database Acknowledge message HTTP response to client
- 36. Zero Latency Logging (ideal) Browser Application Log Server HTTP request Error! HTTP response to client Send log message to database
- 37. What about UDP? ● Yes… Zero latency, but…. ● Do you even care about your logs? (UDP means log messages may get lost...) ● TCP means guaranteed network delivery ● Any non-blocking fails
- 38. Low Latency Logging (balance) Browser Application Log Server HTTP request Error! HTTP response to client Send log message to database
- 39. So how?
- 40. Say hello to RabbitMQ
- 41. What is RabbitMQ? ● ● ● ● Robust messaging for applications Easy to use Runs on all major operating systems Supports a huge number of developer platforms ● Open source and commercially supported www.rabbitmq.com
- 42. RabbitMQ - Basic queue “Consumer” “Publisher” source: http://www.rabbitmq.com/tutorials/tutorial-one-php.html
- 43. RabbitMQ - Exchanges “Publisher” “Publisher” queue 1 “Consumer 1” “Publisher” “Exchange” queue 2 “Consumer 2” “Publisher” “Publisher” source: http://www.rabbitmq.com/tutorials/tutorial-three-php.html
- 44. Using Queues === Fast! Add RabbitMQ to logging architecture...
- 45. Low Latency (using AMQP) EdLogHandlerErrorHandler ->handleException() EdLogLogger ->log() EdLogPublisherAmqpPublisher ->publish() RabbitMQ Logging Server JSON payload
- 46. Low Latency Logging (with AMQP) Browser Application RabbitMQ Log Server HTTP request Error! JSON via AMQP HTTP response Fetch message
- 47. Why bother? ● Scalability Log Worker Application A Log Worker Application B RabbitMQ Log Worker Application C Log Worker
- 48. Single Point of Failure... ● RabbitMQ can do HA RabbitMQ Node 6 RabbitMQ Node 1 RabbitMQ Node 2 RabbitMQ Node 5 RabbitMQ Node 3 RabbitMQ Node 4
- 50. Questions?