Performance and Scalability Testing with Python and Multi-Mechanize
Download as PPT, PDF16 likes9,639 views
Multi-mechanize is a Python framework developed for performance and scalability testing of remote APIs, enabling simultaneous client scripting and workload generation. It allows users to create virtual user scripts to stress test servers and expose application bottlenecks while providing automatic report generation with various statistical measures and graphs. This open-source tool is designed for developers, testers, and operations teams who need to evaluate the performance of their online services.
![Performance and Scalability Testing with Python and Multi-Mechanize Corey Goldberg 2011 web: goldb.org email: [email_address] twitter: @cgoldberg multimechanize.com](https://image.slidesharecdn.com/perfscalepythonmulti-mechanize-110330174840-phpapp02/85/Performance-and-Scalability-Testing-with-Python-and-Multi-Mechanize-1-320.jpg)
![Virtual User Scripting web request with timer and assertions: import mechanize import time class Transaction: def run(self): self.custom_timers = {} br = mechanize.Browser() br.set_handle_robots(False) start_timer = time.time() resp = br.open('http://www.example.com/') resp.read() latency = time.time() - start_timer self.custom_timers['Example_Homepage'] = latency assert (resp.code == 200), 'Bad HTTP Response' assert ('Example Web Page' in resp.get_data()), 'Failed Content Verification'](https://image.slidesharecdn.com/perfscalepythonmulti-mechanize-110330174840-phpapp02/85/Performance-and-Scalability-Testing-with-Python-and-Multi-Mechanize-12-320.jpg)
![Test Configuration sample config file: [global] run_time: 300 rampup: 300 console_logging: off results_ts_interval: 60 [user_group-1] threads: 10 script: search_simple.py [user_group-2] threads: 5 script: search_complex.py](https://image.slidesharecdn.com/perfscalepythonmulti-mechanize-110330174840-phpapp02/85/Performance-and-Scalability-Testing-with-Python-and-Multi-Mechanize-13-320.jpg)
Performance and Scalability Testing with Python and Multi-Mechanize
- 1. Performance and Scalability Testing with Python and Multi-Mechanize Corey Goldberg 2011 web: goldb.org email: [email_address] twitter: @cgoldberg multimechanize.com
- 2. Multi-Mechanize Open Source (LGPLv3) Started Development in 2010 Python 2 (2.6+) Project Hosted on Google Code (SVN) Website: multimechanize.com Download Code Samples / Script Dev Guide Wiki Discussion Group
- 3. What is Multi-Mechanize? "Python framework for performance and load testing remote API's" Enables you to: run simultaneous client scripts to generate workload against a remote service run transactions at an increasing series of load levels against your API and measure performance simulate realistic and projected API usage
- 4. Who is Multi-Mechanize For? Developers Testers Ops ... who need to test and tune the performance and scalability of a web site or API service Assumptions: You know how to code in Python You understand basic performance testing
- 5. Testing for Performance & Scalability You have a remote API or website and need to: stress test server and network infrastructure find application bottlenecks expose errors under load performance benchmarks capacity planning
- 6. Which API's Can You Test? You write the virtual user (client) scripts in Python, so any remote protocol/lib you want: Web/HTTP Raw HTTP (urllib2, httplib) Web Forms (mechanize) REST/SOAP/XMLRPC Sockets Database, NoSQL, Cache Any Python API
- 7. Things I Like Free Runs on multiple platforms (nix, win) Python code base Scripts are in Python (batteries included, python libraries) Scalable load generator Automatic report generation Aggregate stats (response times, throughput, percentiles) Graphs (scatter, time-series)
- 8. Inside Multi-Mechanize Multi-process/Multi-threaded Test Harness Virtual User Scripting in Python Configurable Test Runner Report Generator Input: Virtual User Test Scripts, Config File Output: HTML Report with PNG images (graphs)
- 9. Virtual User Scripting You create test scripts in Python for Multi-Mechanize to run. Each test script must implement a Transaction() class . This class must implement a run() method . So a basic test script consists of: class Transaction: def run(self): # do something here
- 10. Virtual User Scripting During a test run, your Transaction() class is instantiated once, and then its run() method is called repeatedly: class Transaction: def __init__(self): # this gets called once def run(self): # this gets called repeatedly
- 11. Virtual User Scripting A full script that will issue an HTTP GET using urllib2: import urllib2 class Transaction(object): def run(self): url = 'http://www.example.com/' urllib2.urlopen(url).read()
- 12. Virtual User Scripting web request with timer and assertions: import mechanize import time class Transaction: def run(self): self.custom_timers = {} br = mechanize.Browser() br.set_handle_robots(False) start_timer = time.time() resp = br.open('http://www.example.com/') resp.read() latency = time.time() - start_timer self.custom_timers['Example_Homepage'] = latency assert (resp.code == 200), 'Bad HTTP Response' assert ('Example Web Page' in resp.get_data()), 'Failed Content Verification'
- 13. Test Configuration sample config file: [global] run_time: 300 rampup: 300 console_logging: off results_ts_interval: 60 [user_group-1] threads: 10 script: search_simple.py [user_group-2] threads: 5 script: search_complex.py
- 14. Running a Test
- 15. Results Reports
- 16. Results Reports (Stats) test summary transaction timers custom timers (from instrumented client code) time-series/interval data counts rate/throughput response times average, min, max, stdev percentiles (80th, 90th, 95th)
- 17. Results Reports (Graphs) response time scatter plot:
- 18. Results Reports (Graphs) throughput time-series plot:
- 19. Results Reports (Graphs) latency time-series plot:
- 20. The End visit: http://multimechanize.com