Load testing with Blitz
1 like1,271 views
The document discusses load testing and why it often fails. It recommends using a tool with the lowest barrier to entry, such as Blitz, which allows load testing on AWS with a web form or API. Blitz produces results but no reports, so the document shows how to modify Blitz's code to output JSON results for reporting purposes. It encourages integrating load testing into existing development workflows rather than treating it separately after deployment.
![$ blitz curl:help
Usage: blitz curl <options> <url>
--user-agent -A <string> User-Agent to send to server
--cookie -b name=<string> Cookie to send to the server (multiple)
--data -d <string> Data to send in a PUT or POST request
--dump-header -D <file> Print the request/response headers
--referer -e <string> Referer URL
--help -h Help on command line options
--header -H <string> Custom header to pass to server
--pattern -p <s>-<e>:<d> Ramp from s to e concurrent requests in d secs
--region -r <string> california|oregon|virginia|singapore|ireland|japan
--status -s <number> Assert on the HTTP response status code
--timeout -T <ms> Wait time for both connect and responses
--user -u <user[:pass]> User and password for authentication
--request -X <string> Request method to use (GET, HEAD, PUT, etc.)
--variable -v <string> Define a variable to use
--verbose -V Print the request/response headers
--tlsv1 -1 Use TLSv1 (SSL)
--sslv2 -2 Use SSLv2 (SSL)
--sslv3 -3 Use SSLv3 (SSL)](https://image.slidesharecdn.com/load-testing-120524185456-phpapp02/85/Load-testing-with-Blitz-68-320.jpg)
![{"start":1335694346,"finish":1335694367,"results":[{"timestamp":
2.502626,"duration":1.13912,"total":1,"hits":1,"errors":0,"timeouts":
0,"volume":2,"txbytes":258.0,"rxbytes":17810.0,"steps":[{"duration":
2.13912,"connect":0.260004,"errors":0,"timeouts":0,"asserts":0}]},
{"timestamp":5.016102,"duration":0.640047,"total":5,"hits":5,"errors":
0,"timeouts":0,"volume":4,"txbytes":1548.0,"rxbytes":90882.0,"steps":
[{"duration":1.640047,"connect":0.158944,"errors":0,"timeouts":
0,"asserts":0}]},{"timestamp":7.524594,"duration":0.639987,"total":
10,"hits":10,"errors":0,"timeouts":0,"volume":5,"txbytes":
3354.0,"rxbytes":182222.0,"steps":[{"duration":1.639987,"connect":
0.158999,"errors":0,"timeouts":0,"asserts":0}]},{"timestamp":
10.03388,"duration":0.63961,"total":20,"hits":20,"errors":0,"timeouts":
0,"volume":7,"txbytes":5418.0,"rxbytes":364902.0,"steps":[{"duration":
1.63961,"connect":0.158847,"errors":0,"timeouts":0,"asserts":0}]},
{"timestamp":12.543211,"duration":0.640421,"total":31,"hits":
31,"errors":0,"timeouts":0,"volume":9,"txbytes":9030.0,"rxbytes":
565850.0,"steps":[{"duration":1.640422,"connect":0.158767,"errors":
0,"timeouts":0,"asserts":0}]},{"timestamp":15.051705,"duration":
0.639894,"total":44,"hits":44,"errors":0,"timeouts":0,"volume":
10,"txbytes":12384.0,"rxbytes":803334.0,"steps":[{"duration":
1.639894,"connect":0.158794,"errors":0,"timeouts":0,"asserts":0}]},
{"timestamp":17.560136,"duration":-1.0,"total":44,"hits":44,"errors":
0,"timeouts":0,"volume":0,"txbytes":12384.0,"rxbytes":803334.0,"steps":
[{"duration":0.0,"connect":0.0,"errors":0,"timeouts":0,"asserts":0}]}]}](https://image.slidesharecdn.com/load-testing-120524185456-phpapp02/85/Load-testing-with-Blitz-131-320.jpg)
Load testing with Blitz
- 1. Load testing
- 3. Who is this dude?
- 5. BULLETPROOF
- 6. I Run this thing
- 7. Have tried and failed many times to implement effective load testing
- 9. A story
- 10. New project
- 11. Replacing a high- profile website (new infrastructure, new application)
- 13. “We need 100% uptime”
- 14. Project starts
- 15. Requirements change
- 17. Performance testing? Fuck it! We’ll do it live!
- 19. Performance testing? “You have one day”
- 21. Problem:
- 22. Treated as WAterfall-esque QA
- 23. It’s a non-functional requirement
- 25. WHY Do we load test?
- 27. “Can the site deal with load?”
- 28. Defect detection
- 29. “Is this code going to cripple the site?”
- 30. Optimisation
- 31. “What happens if we change this Setting?”
- 33. “Is the site getting slower?”
- 34. Cache warming
- 35. “Prepare the infrastructure for Failover”
- 36. WHY Do we load test?
- 37. A combination of proactive & Reactive reasons
- 38. A combination of proactive & Reactive reasons
- 39. It’s a non-functional requirement
- 41. Why does load testing FAIL? (from a technical perspective)
- 42. How not the WHAT
- 43. TooL fetishism
- 44. Get bogged down in details
- 45. how the testing infrastructure works
- 46. Micro vs Macro
- 47. Business Does not care about your tools
- 49. “Can the site deal with load?”
- 51. Recommendation:
- 52. Use tool with lowest barrier TO entry
- 54. Blitz
- 55. Runs on AWS
- 56. Web form
- 57. Browser plugins Firefox, Chrome
- 58. REST API
- 59. API Clients Ruby, Python, Java Node.JS, Perl
- 61. Excellent documentation docs.blitz.io
- 62. Inexpensive 250 concurrent users over 60 second period == free
- 63. SPRINTS & RUSHES
- 65. Let’s dive in!
- 66. Sign up @ http://blitz.io/
- 67. $ gem install blitz $ blitz help Usage: blitz <command> <options> help - Display this help account:about - Show information about your account api:init - Validate and login with your API key couch:fuzz - Auto generate blitz tests from CouchDB curl - Run a sprint or a rush curl:help - Show help on sprint and rushing traceroute - Run traceroute remotely traceroute:help - Show help on traceroute version - Show the version of this Ruby gem $ blitz api:init # writes credentials to ~/.blitz/credentials
- 68. $ blitz curl:help Usage: blitz curl <options> <url> --user-agent -A <string> User-Agent to send to server --cookie -b name=<string> Cookie to send to the server (multiple) --data -d <string> Data to send in a PUT or POST request --dump-header -D <file> Print the request/response headers --referer -e <string> Referer URL --help -h Help on command line options --header -H <string> Custom header to pass to server --pattern -p <s>-<e>:<d> Ramp from s to e concurrent requests in d secs --region -r <string> california|oregon|virginia|singapore|ireland|japan --status -s <number> Assert on the HTTP response status code --timeout -T <ms> Wait time for both connect and responses --user -u <user[:pass]> User and password for authentication --request -X <string> Request method to use (GET, HEAD, PUT, etc.) --variable -v <string> Define a variable to use --verbose -V Print the request/response headers --tlsv1 -1 Use TLSv1 (SSL) --sslv2 -2 Use SSLv2 (SSL) --sslv3 -3 Use SSLv3 (SSL)
- 70. $ blitz curl --region singapore --pattern 1-250:20 --timeout 10000 --verbose http://example.org/
- 71. user growth / time $ blitz curl --region singapore --pattern 1-250:20 --timeout 10000 --verbose http://example.org/
- 72. user growth / time for AWS lag $ blitz curl --region singapore --pattern 1-250:20 --timeout 10000 --verbose http://example.org/
- 73. user growth / time for AWS lag $ blitz curl --region singapore --pattern 1-250:20 --timeout 10000 --verbose http://example.org/ rushing from singapore... Time Users Response Hits Timeouts Errors Hits/s Mbps 2.5s 31 -1.000s 0 0 0 8.3s 103 1.075s 146 0 9 50.53 0.88 11.2s 140 0.406s 376 0 9 79.64 1.41 14.1s 176 0.409s 682 0 9 105.24 1.86 22.8s 0 0.417s 1552 0 9 15.57 0.27
- 75. blitz calculates the growth automatically --pattern 1-250:20
- 77. TV AD / Twittersplosion --pattern 200-5000:30,5000-400:60
- 79. Integrate with your development cycle
- 80. VCS CI Deploy automation
- 81. VCS CI Deploy automation
- 83. GitHub web hooks
- 84. Doesn’t take into account test failures
- 85. Is the code actually deployed?
- 87. VCS CI Deploy automation
- 88. CRON JOB
- 89. JENKINS JOB
- 90. GREAT if you have a CI system
- 91. Runs tests but Separate system
- 92. What if you’re making infrastructure changes during a load test?
- 94. VCS CI Deploy automation
- 95. Fabric
- 96. Capistrano
- 97. Run tests when you deploy
- 98. Use the same toolchain
- 99. Testing lives IN the app.
- 100. What about triggering load tests on infrastructure changes?
- 101. MVP
- 104. Problem:
- 105. No REporting :-(
- 106. THIS IS IT rushing from singapore... Time Users Response Hits Timeouts Errors Hits/s Mbps 2.5s 31 -1.000s 0 0 0 8.3s 103 1.075s 146 0 9 50.53 0.88 11.2s 140 0.406s 376 0 9 79.64 1.41 14.1s 176 0.409s 682 0 9 105.24 1.86 22.8s 0 0.417s 1552 0 9 15.57 0.27
- 107. SOL ON:
- 108. Optimisation
- 109. “What happens if we change this Setting?”
- 110. Historical analysis
- 111. “Is the site getting slower?”
- 112. Showstopper?
- 113. ...
- 114. $ gem install blitz # ... rushing from singapore... Time Users Response Hits Timeouts Errors Hits/s Mbps 2.5s 31 -1.000s 0 0 0 8.3s 103 1.075s 146 0 9 50.53 0.88 11.2s 140 0.406s 376 0 9 79.64 1.41 14.1s 176 0.409s 682 0 9 105.24 1.86 22.8s 0 0.417s 1552 0 9 15.57 0.27
- 115. Can we duck punch our way to success?
- 117. #!/usr/bin/env ruby # load_test.rb require 'blitz' arguments = "--region singapore --pattern 1:250:60" url = "http://example.org/" command = "#{arguments} #{url}" job = Blitz::Curl.parse(command) curl = Blitz::Command::Curl.new curl.rush(job)
- 118. # ruby load_test.rb rushing from singapore... Time Users Response Hits Timeouts Errors Hits/s Mbps 2.5s 31 -1.000s 0 0 0 8.3s 103 1.075s 146 0 9 50.53 0.88 11.2s 140 0.406s 376 0 9 79.64 1.41 14.1s 176 0.409s 682 0 9 105.24 1.86 22.8s 0 0.417s 1552 0 9 15.57 0.27
- 119. #!/usr/bin/env ruby # load_test.rb require 'blitz' arguments = "--region singapore --pattern 1:250:60" url = "http://example.org/" command = "#{arguments} #{url}" job = Blitz::Curl.parse(command) curl = Blitz::Command::Curl.new curl.rush(job)
- 120. #!/usr/bin/env ruby # load_test.rb require 'blitz' arguments = "--region singapore --pattern 1:250:60" url = "http://example.org/" command = "#{arguments} #{url}" job = Blitz::Curl.parse(command) curl = Blitz::Command::Curl.new curl.rush(job)
- 121. Blitz::Curl.parse(command) returns Blitz::Curl::Rush
- 122. #!/usr/bin/env ruby # load_test.rb require 'blitz' arguments = "--region singapore --pattern 1:250:60" url = "http://example.org/" command = "#{arguments} #{url}" job = Blitz::Curl.parse(command) curl = Blitz::Command::Curl.new curl.rush(job)
- 124. initiates the test + outputs the test results Blitz::Command::Curl#rush
- 125. #!/usr/bin/env ruby # load_test.rb require 'blitz' arguments = "--region singapore --pattern 1:250:60" url = "http://example.org/" command = "#{arguments} #{url}" job = Blitz::Curl.parse(command) curl = Blitz::Command::Curl.new curl.rush(job) puts job.result.timeline.to_json
- 126. Produces muck
- 127. duck punch our way to success
- 128. class Blitz::Curl::Rush class Point def to_json(*args) { 'timestamp' => @timestamp, 'duration' => @duration, 'total' => @total, 'hits' => @hits, 'errors' => @errors, 'timeouts' => @timeouts, 'volume' => @volume, 'txbytes' => @txbytes, 'rxbytes' => @rxbytes, 'steps' => @steps, }.to_json end end # ...
- 129. # ... class Step def to_json(*args) { 'duration' => @duration, 'connect' => @connect, 'errors' => @errors, 'timeouts' => @timeouts, 'asserts' => @asserts, }.to_json end end end
- 130. #!/usr/bin/env ruby # load_test.rb require 'blitz' arguments = "--region singapore --pattern 1:250:60" url = "http://example.org/" command = "#{arguments} #{url}" job = Blitz::Curl.parse(command) curl = Blitz::Command::Curl.new curl.rush(job) puts job.result.timeline.to_json
- 132. Do with as you please
- 133. Here’s one I prepared earlier..
- 135. ...
- 136. You don’t have to set up a tool
- 137. But you have to write reporting?
- 138. At least you have load testing?
- 139. It’s on the way..
- 142. Where to from here?
- 143. Pull in data from other places
- 144. new relic Ganglia/collectd Git commits
- 147. Don’t like Blitz?
- 148. SaaS Load impact Browser mob
- 149. Open Source Tsung JMeter
- 152. Low barrier of entry is key
- 153. Integrate with your existing dev tools
- 155. Questions?