distributed message queue dengan python celery dan rabbitmq
Download as ODP, PDF0 likes149 views
This document discusses using Python Celery and RabbitMQ to create a distributed message queue. It describes common problems that message queues can help with like long-running tasks and tasks that depend on third parties. It provides an overview of message brokers, exchanges, bindings and queues. It then demonstrates setting up Celery with RabbitMQ and running simple tasks asynchronously and monitoring their progress.
![from celery import Celery
app = Celery('proj',
broker='amqp://',
backend='rpc://',
include=['proj.tasks'])
# Optional configuration
app.conf.update(
result_expires=3600,
)
if __name__ == '__main__':
app.start()
Proj/celery.py](https://image.slidesharecdn.com/presentasitanpajudul-170402165811/85/distributed-message-queue-dengan-python-celery-dan-rabbitmq-26-320.jpg)
![from proj.celery import app
if __name__ == '__main__':
add_result = []
multiply_result = []
for y in range(1, 50):
for x in range(1, 4):
add_result.append(app.send_task('proj.tasks.add', [x, x]))
multiply_result.append(app.send_task('proj.tasks.multiply', [x, x]))
for index, async_result in enumerate(add_result):
print("state" + str(index) + " = " + str(async_result.state))
print("state" + str(index) + " = " + str(async_result.result))
for index, async_result in enumerate(multiply_result):
print("state" + str(index) + " = " + str(async_result.state))
print("state" + str(index) + " = " + str(async_result.result))
producer.py](https://image.slidesharecdn.com/presentasitanpajudul-170402165811/85/distributed-message-queue-dengan-python-celery-dan-rabbitmq-28-320.jpg)
distributed message queue dengan python celery dan rabbitmq
- 1. DISTRIBUTED MESSAGE QUEUE DENGAN PYTHON CELERY DAN RABBITMQ
- 2. Problem 1. Ada tugas yang membutuhkan waktu lama 2. Ada tugas yang bergantung pada pihak ketiga dan harus diulang ketika gagal (e.g mengirim email verifikasi)
- 3. Eksperimen tahap pertama 1. Cron job 2. CLI Script 3. Lakukan dengan array
- 4. Hasil Eksperimen Tahap Pertama 1. Background-job susah di monitor 2. Background-job susah di scale
- 5. STANDING ON THE SHOULDERS OF GIANTS Memakai message broker dan task queue library yang sudah ada Eksperimen Tahap Kedua
- 6. Message broker 1. RabbitMQ 2. Beanstalk 3. Gearman 4. dll
- 7. 1. Dokumentasinya bagus 2. Mudah dapat bahan referensi dari google 3. OSS dan komersial support
- 9. Messaging Concept Producer, Consumer, Broker Message Queue
- 10. P CBROKER
- 11. AMQP Advanced Message Queueing Protocol
- 13. Exchange types
- 16. Binding, aturan yang digunakan Exchange untuk menentukan Route ke Queue.
- 19. (consumer) hello_world.py from celery import Celery app = Celery('print-hello-world', backend='rpc', broker='amqp://') @app.task def print_hello(some_words='world'): return 'hello {0}.'.format(some_words)
- 20. Starting Celery in terminal celery worker -A hello_world
- 22. Producer from hello_world import print_hello add_new_message = print_hello.delay(some_words=’a message’)
- 23. Multi Celery Process celery worker -A hello_world -n <process_name>
- 25. Second example ├── producer.py ├── proj ├── celery.py ├── __init__.py └── tasks.py
- 26. from celery import Celery app = Celery('proj', broker='amqp://', backend='rpc://', include=['proj.tasks']) # Optional configuration app.conf.update( result_expires=3600, ) if __name__ == '__main__': app.start() Proj/celery.py
- 27. from .celery import app @app.task def add(x, y): return x + y @app.task def multiply(x, y): return x * y Proj/tasks.py
- 28. from proj.celery import app if __name__ == '__main__': add_result = [] multiply_result = [] for y in range(1, 50): for x in range(1, 4): add_result.append(app.send_task('proj.tasks.add', [x, x])) multiply_result.append(app.send_task('proj.tasks.multiply', [x, x])) for index, async_result in enumerate(add_result): print("state" + str(index) + " = " + str(async_result.state)) print("state" + str(index) + " = " + str(async_result.result)) for index, async_result in enumerate(multiply_result): print("state" + str(index) + " = " + str(async_result.state)) print("state" + str(index) + " = " + str(async_result.result)) producer.py