Automatic testing and quality assurance for WordPress plugins and themes
3 likes1,168 views
The document discusses automatic testing and quality assurance for WordPress plugins and themes, emphasizing the importance of utilizing tools like PHP Code Sniffer, PHPUnit, and continuous integration platforms to ensure code quality. It presents various methods for automating tests and quality checks, such as using Git hooks and Travis CI, alongside best practices for performance monitoring and visual regression testing. The ultimate goal is to enhance the reputation of WordPress plugins by demonstrating reliable quality through automation in the development process.
![Example .git/hooks/pre-commit
# Loop all files that are about to be committed (diff of git head and staged)
echo "==> Checking syntax errors..."
for FILE in $(git diff --cached --name-only); do
resource="$REPO_DIR/$FILE"
##
# Test PHP syntax for all changed *.php and *.module files
##
if [[ "$FILE" =~ ^.+(php|module)$ ]]; then
if [[ -f $resource ]]; then
phpcs "$resource" 1> /dev/null
if [ $? -ne 0 ]; then
errors+=("PHP syntax Error: $FILE")
fi
fi
fi
done
See code at seravo.com/coding-wordpress-in-style-with-phpcs](https://image.slidesharecdn.com/automatictestingandqaforwordpressplugins-181111192039/85/Automatic-testing-and-quality-assurance-for-WordPress-plugins-and-themes-23-320.jpg)
![Example .travis.yml
before_install:
- if [[ "$SNIFF" == "1" ]]; then git clone …
script:
# Syntax check all php files and fail for any error text in STDERR
- '! find . -type f -name "*.php" -exec
php -d error_reporting=32767 -l {} ; 2>&1 >&- | grep "^"'
# More extensive PHP Style Check
- if [[ "$SNIFF" == "1" ]]; then $PHPCS_DIR/bin/phpcs -i;
$PHPCS_DIR/bin/phpcs --standard=phpcs.xml; fi
- phpunit](https://image.slidesharecdn.com/automatictestingandqaforwordpressplugins-181111192039/85/Automatic-testing-and-quality-assurance-for-WordPress-plugins-and-themes-29-320.jpg)
![Measure execution time and memory
echo "<!-- Measurements: ";
echo memory_get_usage();
echo " - ";
echo (microtime(true) - $_SERVER["REQUEST_TIME_FLOAT"]);
echo " -->";
$ for x in {1..20}
do curl -s http://localhost | grep "Measurements"
done
<!-- Measurements: 5761720 - 0.2341411113739
<!-- Measurements: 5761720 - 0.24964690208435
<!-- Measurements: 5761704 - 0.25908708572388
<!-- Measurements: 5761720 - 0.23540115356445
...](https://image.slidesharecdn.com/automatictestingandqaforwordpressplugins-181111192039/85/Automatic-testing-and-quality-assurance-for-WordPress-plugins-and-themes-32-320.jpg)
Automatic testing and quality assurance for WordPress plugins and themes
- 1. Automatic testing and quality assurance for WordPress plugins and themes WP Helsinki Meetup 7.11.2018 Otto Kekäläinen @ottokekalainen WP-palvelu.fi / Seravo.com
- 2. ● WP-palvelu.fi – WordPress hosting and upkeep ● CEO, sysadmin and developer ● Linux and open source advocate ● Contributed to WordPress Core, fi and sv translations, Linux, Docker, Nginx, Redis, MariaDB… ● Twitter:@ottokekalainen Otto Kekäläinen
- 3. Enterprise grade hosting and upkeep for WordPress
- 4. FIRST THINGS FIRST ● Before you write any WordPress theme or plugin code, please read up on the basics at: ○ developer.wordpress.org/themes ○ developer.wordpress.org/plugins
- 5. CHALLENGE: EVERYBODY WANTS QUALITY Developers just don’t have enough time or customers budget
- 6. Solution: Automatic ~ zero cost
- 7. DO EVERYTHING THAT CAN BE AUTOMATED ● scan code to find errors ○ static analysis ● run code to find errors ○ unit and integration tests
- 8. TOOLS 1. What can test PHP code 2. What can automate tests
- 9. WHAT TO DO ABOUT PHP CODE ● PHP Code Sniffer ● PHP unit tests ● PhantomJS Headless Chrome integration tests ● performance ○ execution time ○ memory usage
- 10. PHP CODE SNIFFER
- 11. PHPCS
- 12. PHPCFB AND GIT CITOOL
- 13. phpcs.xml <?xml version="1.0"?> <ruleset name="Seravo"> <!-- show progress --> <arg value="p"/> <!-- check current and all subfolders if no file parameter given --> <file>.</file> <rule ref="Squiz.PHP.CommentedOutCode"/> <rule ref="Squiz.WhiteSpace.SuperfluousWhitespace"/> <rule ref="Generic.CodeAnalysis.UnusedFunctionParameter"/> <rule ref="Generic.Commenting.Todo"/> <rule ref="Generic.ControlStructures.InlineControlStructure"/> <rule ref="WordPress-Extra"> <exclude name="Generic.WhiteSpace.DisallowSpaceIndent"/> <exclude name="Generic.WhiteSpace.ScopeIndent"/> <exclude name="WordPress.WhiteSpace.PrecisionAlignment.Found" /> <exclude name="WordPress.PHP.YodaConditions" /> </rule> </ruleset>
- 14. phpcs --standard=Security FILE: tools/gapi.php --------------------------------------------------------------------- FOUND 1 ERROR AND 8 WARNINGS AFFECTING 8 LINES --------------------------------------------------------------------- 35 | WARNING | Possible RFI detected with GADWP_DIR on include_once 51 | WARNING | Function array_map() that supports callback detected 148 | WARNING | Possible XSS detected with esc_url on echo 152 | WARNING | Possible XSS detected with __ on echo 156 | WARNING | Possible XSS detected with _e on echo 307 | WARNING | Crypto function crc32 used. 767 | WARNING | Function array_map() that supports callback --------------------------------------------------------------------- More tips at seravo.com/coding-wordpress-in-style-with-phpcs
- 15. PHP UNIT TESTS assertEquals(a, b) Lots of PHP Unit test examples in WordPress Core source
- 17. Example test code <?php class ExampleCest { /** * Open front page (/) **/ public function openFrontPage(AcceptanceTester $I) { $I->amOnPage('/'); $I->checkBrowserConsole(); $I->see('WordPress'); } } Read more at seravo.com/docs/tests/ng-integration-tests
- 18. VISUAL REGRESSION TESTS $ gm compare -highlight-style assign -highlight-color purple -file diff.png *.png
- 19. VISUAL REGRESSION TESTS $ gm compare -verbose -metric mse *.png Image Difference (MeanSquaredError): Normalized Absolute ============ ========== Red: 0.0319159868 8.1 Green: 0.0251841368 6.4 Blue: 0.0278537225 7.1 Opacity: 0.0000000000 0.0 Total: 0.0212384615 5.4
- 20. Where do you draw the line between acceptable changes and failures/regressions?
- 21. HOW TO AUTOMATE ● git pre-commit hook ○ local tests ● git receive hook on a remote server ○ Github + Travis-CI ○ Gitlab + Gitlab-CI ○ Bitbucket ○ etc..
- 22. GIT PRE-COMMIT HOOK IN ACTION
- 23. Example .git/hooks/pre-commit # Loop all files that are about to be committed (diff of git head and staged) echo "==> Checking syntax errors..." for FILE in $(git diff --cached --name-only); do resource="$REPO_DIR/$FILE" ## # Test PHP syntax for all changed *.php and *.module files ## if [[ "$FILE" =~ ^.+(php|module)$ ]]; then if [[ -f $resource ]]; then phpcs "$resource" 1> /dev/null if [ $? -ne 0 ]; then errors+=("PHP syntax Error: $FILE") fi fi fi done See code at seravo.com/coding-wordpress-in-style-with-phpcs
- 26. TRAVIS-CI CHECKING EVERY COMMIT
- 28. NOTIFICATION EMAILS THAT CAN’T GO UNNOTICED
- 29. Example .travis.yml before_install: - if [[ "$SNIFF" == "1" ]]; then git clone … script: # Syntax check all php files and fail for any error text in STDERR - '! find . -type f -name "*.php" -exec php -d error_reporting=32767 -l {} ; 2>&1 >&- | grep "^"' # More extensive PHP Style Check - if [[ "$SNIFF" == "1" ]]; then $PHPCS_DIR/bin/phpcs -i; $PHPCS_DIR/bin/phpcs --standard=phpcs.xml; fi - phpunit
- 30. Live examples github.com/Seravo/seravo-plugin github.com/Seravo/linux-tuki.fi travis-ci.org/Seravo
- 31. FREE FOR OPEN SOURCE CODE SERVICES ● circleci.com ● cocodacy.com ● codeclimate.com ● codeship.com ● coveralls.io ● coverity.com ● sourceclear.com ● travis-ci.org (.com for private repos) Listed in alphabetic order, no preference.
- 32. Measure execution time and memory echo "<!-- Measurements: "; echo memory_get_usage(); echo " - "; echo (microtime(true) - $_SERVER["REQUEST_TIME_FLOAT"]); echo " -->"; $ for x in {1..20} do curl -s http://localhost | grep "Measurements" done <!-- Measurements: 5761720 - 0.2341411113739 <!-- Measurements: 5761720 - 0.24964690208435 <!-- Measurements: 5761704 - 0.25908708572388 <!-- Measurements: 5761720 - 0.23540115356445 ...
- 33. Test with dummy data ● While developing a site, load lots of dummy data into it so you can test how your site looks and performs with 100, 1000 or 100 000 posts. ● Basic: Import themeunittestdata.wordpress.xml ○ codex.wordpress.org/Theme_Unit_Test ● More data: wp post generate ○ curl http://loripsum.net/api/5 | wp post generate --post_content --count=10 ● More realism: wp-cli-fixtures ○ github.com/nlemoine/wp-cli-fixtures
- 34. WordPress plugins have a reputation of low quality. Help us prove them wrong. Start using automatic quality testing!
- 35. Hopefully automatic quality testing will be integrated into the WordPress.org plugin directory in the future. See make.wordpress.org/tide
- 36. Extra tip: use the WP plugin boiler plate to start with: wppb.io WP Theme starter example: github.com/aucor/aucor-starter
- 37. This presentation was about plugin and theme development. How about testing a real WordPress site to ensure updates don’t break it? Let Seravo handle updates of production sites for you. See our hosting and upkeep service at Seravo.com