マイクロサービスを利用する側のパフォーマンス向上策 (PyCon JP 2016)
4 likes2,744 views
The document discusses how to improve application performance using asynchronous and concurrent programming with gevent. It provides examples of patching urllib2 to enable asynchronous requests, spawning multiple requests concurrently using gevent, and limiting concurrent requests using a semaphore to control throughput. It also shows how to run a WSGI server asynchronously using gevent to handle requests without blocking.
![https://www.monotaro.com/
API
[ms]](https://image.slidesharecdn.com/pyconjp2016-microservice-client-performance1-160921045835/85/PyCon-JP-2016-36-320.jpg)
マイクロサービスを利用する側のパフォーマンス向上策 (PyCon JP 2016)
- 22. https://www.monotaro.com/ 22 Ordered by: cumulative time ncalls tottime percall cumtime percall filename:lineno(function) 1 0.000 0.000 3.330 3.330 run_web.py:134(application) 1 0.000 0.000 3.330 3.330 (フレームワーク初期化) 1 0.000 0.000 3.249 3.249 (フレームワーク初期化) 1 0.000 0.000 3.180 3.180 (フレームワーク初期化) 1 0.000 0.000 3.180 3.180 (画面実装部分) 19 0.000 0.000 2.991 0.157 (画面実装部分) 19 0.000 0.000 2.920 0.154 (画面実装部分) 10 0.000 0.000 2.865 0.286 (画面実装部分) 7591 2.784 0.000 2.784 0.000 {method 'recv' of '_socket.socket' objects} 19 0.000 0.000 2.275 0.120 liburllib2.py:122(urlopen) 19 0.000 0.000 2.275 0.120 liburllib2.py:373(open) 19 0.000 0.000 2.274 0.120 liburllib2.py:401(_open) (略)
- 24. https://www.monotaro.com/ 24 Boost application performance using asynchronous I/O http://www.ibm.com/developerworks/library/l-async/
- 25. https://www.monotaro.com/ 25 Boost application performance using asynchronous I/O http://www.ibm.com/developerworks/library/l-async/
- 28. https://www.monotaro.com/ 28 # 標準ライブラリにモンキーパッチ # 必ず標準ライブラリの前に書くこと！ from gevent import monkey monkey.patch_all() # urllib2が非同期IO版に置換される import urllib2 import gevent
- 29. https://www.monotaro.com/ 29 def get_json(self, url, params): response = urllib2.urlopen(url).read() return json.loads(response) # 変更前 (urlopenで待たされる) result = self.get_json(any_url, any_params)
- 30. https://www.monotaro.com/ 30 def get_json(self, url, params): response = urllib2.urlopen(url).read() return json.loads(response) # get_jsonはすぐには実行しない future = gevent.spawn(self.get_json, any_url, any_params) (他の処理を実行できる) # future.get()実行結果が来るまでブロック result = future.get()
- 31. https://www.monotaro.com/ 31 future1 = gevent.spawn(self.get_json, url1, param1) future2 = gevent.spawn(self.get_json, url2, param2) future3 = gevent.spawn(self.get_json, url3, param3) future4 = gevent.spawn(self.get_json, url4, param4) future5 = gevent.spawn(self.get_json, url5, param5) …
- 32. https://www.monotaro.com/ 32 from gevent.lock import Semaphore # セマフォを用いて同時処理数を50に制限 semaphore = Semaphore(50) def get_json(self, url, params): # このコンテキスト内には同時に最大50しか入れない with semaphore: response = urllib2.urlopen(url).read() return json.loads(response)
- 33. https://www.monotaro.com/ 33 from gevent.pywsgi import WSGIServer http_server = WSGIServer(('', 5000), app) http_server.serve_forever()