Magento Testing on all fronts
12 likes3,585 views
This document discusses various testing strategies and tools for Magento, including: - Unit, integration, acceptance, and load testing approaches using tools like PHPUnit, Selenium, and JMeter - Best practices for continuous integration like version control, code reviews, and automation - Specific frameworks and libraries that can be used for testing Magento like EcomDev PHPUnit and Menta
![Menta for PHPUnit
Text Result PrinterPHPUnit 3.7.20 by Sebastian Bergmann.
Configuration read from /var/www/Tests/Acceptance/conf/devfb.ff.vmhost.xml
> SUITE:
> SUITE:
> SUITE: Acceptance_Tests_Account_Register
> TEST: Acceptance_Tests_Account_Register::register
SUCCESS. (Duration: 8.18 sec)
> TEST: Acceptance_Tests_Account_Register::login
SUCCESS. (Duration: 6.5 sec)
> TEST: Acceptance_Tests_Account_Register::testRegistrationMail
FAILURE: Searching for mail with subject 'Welcome, Test_705b077a962977e1a412696ea42249ae User_705b077a962977e1a412696ea42249ae' timed out (Duration: 22
< Duration: 36 sec
< Duration: 36 sec
> SUITE:
> SUITE: Acceptance_Tests_Checkout_CheckoutMultiAddressTest
> TEST: Acceptance_Tests_Checkout_CheckoutMultiAddressTest::putProductsInCart
FAILURE: Element "//table[@id="shopping-cart-table"]//h3[contains(concat(' ', @class, ' '), ' attentionText ')]" does not contain text "Bulb Vase" Fail
< Duration: 9.87 sec
> SUITE: Acceptance_Tests_Checkout_CheckoutTest
> TEST: Acceptance_Tests_Checkout_CheckoutTest::categoryView
SUCCESS. (Duration: 1.79 sec)
> TEST: Acceptance_Tests_Checkout_CheckoutTest::putPlatterInCart
FAILURE: Failed asserting that two strings are equal. (Duration: 3.55 sec)
SKIPPED: This test depends on "Acceptance_Tests_Checkout_CheckoutTest::putPlatterInCart" to pass. (Duration: 5.34 sec)](https://image.slidesharecdn.com/2014-03magentomeetup-testing-export-140313130730-phpapp01/85/Magento-Testing-on-all-fronts-52-320.jpg)
Magento Testing on all fronts
- 1. On all Fronts Testing Magento San Francisco Magento Meetup Group - March 11, 2014
- 3. E-Commerce: Magento CMS: TYPO3 Portals: ZF, FLOW,… Mobile Searchperience: SOLR 120 people in 7 offices world-wide High Performance /Scale Global Enterprise Projects
- 4. Unit Service UI Acceptance http://www.mountaingoatsoftware.com/blog/the-forgotten-layer-of-the-test-automation-pyramid What the customer thinks the code should do What the developer thinks the code should do Load Test
- 6. Quality Assurance Continuous Everything sounds better with “continuous”
- 7. “Continuous” ==“Doingstuffoverandoveragain” Builds Deployment Tests Infrastructure Setup Devbox Setup == “Automation”
- 8. timespent task size does it manually does it manually gets annoyed writes script to automate runs script wins loses Geeks vs. Non-Geeks Makes fun of geek’s complicated method
- 9. — Ovidiu Platon We are not shipping your machine! if it works on your machine! I don’t care
- 10. Picture The BIG
- 11. Vagrant Boxes
- 12. Awesome stuff Every single line potentially can break production! Bratwurst (Germans) Donut (Americans) Pizza (Italians)
- 13. Version Control GIT, SVN,… PHP lint
- 14. Version Control Code Reviews Gerrit, Crucible, Phabricator,…
- 15. Code Reviews Build Files DB Installer Settings Triggered by commit, by time or manually Jenkins minify js/css .tar.gz “virtual package” (revision)
- 16. Version Control Code Reviews Build Files DB Installer Settings
- 18. Static Code Analysis Install on “Integration” First system where code from all developers/teams meets each other unstable system. Might be broken. Will be rebuild without warning
- 20. Unit Tests Integration Tests Test features that rely on other parts (e.g. framework, external services,…) External components (SSL, Redis, Varnish,…) Basically this is also PHPUnit, but addressing other stuff
- 21. Integration Tests Acceptance Tests
- 22. Acceptance Tests Install on “Staging” Behat (Cucumber) Same setup as production, but different machines Client QA / approval
- 23. Install on “Staging” *http://www.slideshare.net/aoemedia/performance-measurement-and-tuning Stress Tests Jmeter Spin up EC2 instances in the cloud to produce traffic*
- 26. Version Control Code Reviews Build Static Code Analysis Install on “Integration” Unit Tests Integration Tests Acceptance Tests Install on “Staging” Install on “Production” Enable Features
- 29. Config Model Block Helper Controller fat model, skinny controller…
- 31. Assertions
- 32. Fixtures
- 33. Mocks
- 34. Data Provider
- 35. Magento Integration Testsuite management Framework integration Create test environment (db+config)
- 38. Acceptance Tests
- 39. What’s out there? Selenium for PHP PHPUnit_Selenium (Sebastian Bergmann) WebDriver-PHP (Chibimagic) Php-webdriver-bindings (Lukasz Kolczynski) php-webdriver PHPUnit_Selenium (Giorgio Sironi)
- 40. php-webdriver Selenium 2 08/2011 Facebook/php-webdriver (Justin Bishop) Element-34/php-webdriver (Adam Goucher) instaclick/php-webdriver (Anthon Pang)
- 41. php-webdriver “A very thin wrapper of WebDriver”
- 42. So what? A thin layer good! But not enough… no test logic no higher level components dealing with Selenium specific tasks not compatible to legacy test cases No abstraction / structure
- 43. Architecture HTTP AUT No direct access (DB, classes,…) REST* (over HTTP) *http://code.google.com/p/selenium/wiki/JsonWireProtocol
- 45. Project test cases PHPUnit Project Components Components Library Menta php-webdriver reusable project specific
- 46. Menta Selenium 2 Testing Framework for PHP Who is “Menta”? What is “Menta”?
- 47. Features MentaThe framework. Not the cat… Components Session Management ?Bonus Features :)Event / ObserverConfiguration Management PHPUnit integration
- 48. Why? ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ Data Providers ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ Dependent Tests ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ Test Infrastructure
- 49. Menta for PHPUnit Abstract test class PHPUnit_Framework_TestCaseMenta_PHPUnit_Testcase_Selenium2ProjectY_Abstract_TestCase Very thin layer on top of PHPUnit ProjectY_TestCase_Registration ProjectY_TestCase_Checkout the “magic” happens in the components
- 50. HTML ReportScreenshots (on demand and on failure) test description from phpdoc additional messages error details
- 52. Menta for PHPUnit Text Result PrinterPHPUnit 3.7.20 by Sebastian Bergmann. Configuration read from /var/www/Tests/Acceptance/conf/devfb.ff.vmhost.xml > SUITE: > SUITE: > SUITE: Acceptance_Tests_Account_Register > TEST: Acceptance_Tests_Account_Register::register SUCCESS. (Duration: 8.18 sec) > TEST: Acceptance_Tests_Account_Register::login SUCCESS. (Duration: 6.5 sec) > TEST: Acceptance_Tests_Account_Register::testRegistrationMail FAILURE: Searching for mail with subject 'Welcome, Test_705b077a962977e1a412696ea42249ae User_705b077a962977e1a412696ea42249ae' timed out (Duration: 22 < Duration: 36 sec < Duration: 36 sec > SUITE: > SUITE: Acceptance_Tests_Checkout_CheckoutMultiAddressTest > TEST: Acceptance_Tests_Checkout_CheckoutMultiAddressTest::putProductsInCart FAILURE: Element "//table[@id="shopping-cart-table"]//h3[contains(concat(' ', @class, ' '), ' attentionText ')]" does not contain text "Bulb Vase" Fail < Duration: 9.87 sec > SUITE: Acceptance_Tests_Checkout_CheckoutTest > TEST: Acceptance_Tests_Checkout_CheckoutTest::categoryView SUCCESS. (Duration: 1.79 sec) > TEST: Acceptance_Tests_Checkout_CheckoutTest::putPlatterInCart FAILURE: Failed asserting that two strings are equal. (Duration: 3.55 sec) SKIPPED: This test depends on "Acceptance_Tests_Checkout_CheckoutTest::putPlatterInCart" to pass. (Duration: 5.34 sec)
- 53. Components Common Assert Wait Screenshot Selenium1Facade Imap WebService W3C Validator Curl … Some might not even require a Selenium session
- 54. Components! PageObjects? PageObjectsComponents { }}⊇{ PageObjects are components
- 55. PageObjects? represents services offered by a page or feature is the only thing with a deep knowledge of the HTML clickCheckoutButton() isInCart($product) getTotals() ✔ ✔ ✔
- 58. Perceptual diffs Build X Build X-1 Perceptual Diff
- 59. github.com / AOEmedia / Menta github.com / AOEmedia / Menta_SampleProject It’s composer based and you’ can get started in a few minutes!
- 66. Creating JMeter tests JMeter Test Manually Replaying Apache Logs Recording Browser interaction using a proxy“Convert” existing acceptance tests CSV Files (Selenium) HTT
- 67. HTT Behat Selenium Firefox Proxy JMeter Test Converting Behat to JMeter
- 69. Using a Proxy Browser configuration Manual configuration Browser Extension
- 70. Using a Proxy Browser configuration
- 71. java -jar selenium-server-standalone-2.40.0.jar -trustAllSSLCertificates -Dwebdriver.firefox.profile=jmeter Using a Proxy Selenium configuration
- 73. https://jmeter.apache.org/usermanual/test_plan.html • ThreadGroups • Controllers • Samplers • Logic Controllers • Test Fragments • Timers • Assertions • Configuration Elements • Pre-Processor Elements • Post-Processor Elements Elements of a Test Plan
- 74. http://www.slideshare.net/shubelal/abj-meterbrief-upladed/7 Elements of a Test Plan
- 75. • Tree structure • Inheritance (Cookie Manager, Header Manager,…) • Timers • Variables • Assertions • Random values • Scripting • Extracting values • Following redirects • (check examples in bin/examples) Basic Concepts
- 76. Workflows Search / Filters Static Pages 404 What to testConsider conversion rate / funnel!
- 79. Distributed load testing General Master/Slave setup https://jmeter.apache.org/usermanual/jmeter_distributed_testing_step_by_step.pdf AOE presentations http://www.slideshare.net/axeljung01/jmeter-and-amazon-cloud-workshop-t3dd2012 http://www.slideshare.net/aoemedia/performance-measurement-and-tuning On EC2 https://github.com/oliverlloyd/jmeter-ec2 http://www.http503.com/2012/run-jmeter-on-amazon-ec2-cloud/