SlideShare a Scribd company logo
Backstop JS
Visual Testing
1
I would define visual testing as using a headless browser to capture and
compare images of components of a website.
This can be used to implement unit tests for how the CSS is rendering the HTML
in a browser.
Not intended to test how the HTML is generated or replace Unit tests in a back
end service layer.
Not intended to replace cross browser testing.
Allows comparison between sites or comparing against a reference image
captured form the same site before changes were made.
What do you mean by Visual
Testing ?
1. Chrome Headless
2. Setup and Examples
3. What to test
4. Setup in your project
Contents :
Chrome Headless
3
What has changed ?
Why is this a big deal ?
4
 Headless Chrome is shipping in Chrome 59. It's a way to run the Chrome
browser in a headless environment. Essentially, running Chrome without
chrome! It brings all modern web platform features provided by Chromium
and the Blink rendering engine to the command line.
Why is that useful?
A headless browser is a great tool for automated testing and server
environments where you don't need a visible UI shell. For example, you may
want to run some tests against a real web page, create a PDF of it, or just
inspect how the browser renders an URL.
Announcement blog :
https://developers.google.com/web/updates/2017/04/headless-chrome
How to Get it:
As long as you have a version of chrome installed greater than Chrome 59 –
( Chrome 61 on Windows) you already have it 
Chrome Headless
5
 BackstopJS automates visual regression testing of your responsive web UI by
comparing DOM screenshots over time.
Backstop is a open source project to run visual tests using headless browsers to
capture screenshots.
Originally ran using either PhantomJS or SlimerJS headless browser libraries.
These were implementations of WebKit (Chrome) or Gecko (Firefox).
From release v3.0.19 this now supports Chrome headless.
The latest release has a lot of stability improvements v3.0.27
Git Hub URL : https://github.com/garris/BackstopJS
Features :
• Allows capture and comparison of screenshots form headless browser
• Excellent comparison tool to identify and highlight differences
• Allows setup of multiple breakpoints to test responsive sites
• Uses simple CSS selectors to identify what to capture
• Pre capture scripts can be enabled to be ran
Backstop JS
6
Chrome is based on WebKit but using PhantomJS as a headless browser
caused issues making it unusable.
There would be styling issues that would be thrown that would not be present on
the site when viewed through chrome.
As visual testing is about how the site renders this causes the tests to be
unreliable.
When I investigated implementing visual testing this was the stumbling block
each time.
Chrome != Webkit
Setup and Examples
7
8
• backstop init : Set up a new BackstopJS instance -- specify URLs, cookies,
screen sizes, DOM selectors, interactions etc. (see examples directory)
• backstop test : BackstopJS creates a set of test screenshots and compares
them with your reference screenshots. Any changes show up in a visual
report. (Run this after making CSS changes as many times as needed.)
• backstop approve : If the test you ran looks good, then go ahead and
approve it. Approving changes will update your reference files with the
results from your last test. Future tests are compared against your most
recent approved test screenshots.
The BackstopJS workflow
9
backstop init
10
backstop test (initial)
11
backstop approve
12
backstop test
13
{
"id": "backstop_default",
"viewports": [
{
"label": "phone",
"width": 320,
"height": 480
},
{
"label": "tablet",
"width": 1024,
"height": 768
}
],
"onBeforeScript": "chromy/onBefore.js",
"onReadyScript": "chromy/onReady.js",
"scenarios": [
…………
],
"paths": {
"bitmaps_reference": "backstop_data/bitmaps_reference",
"bitmaps_test": "backstop_data/bitmaps_test",
"engine_scripts": "backstop_data/engine_scripts",
"html_report": "backstop_data/html_report",
"ci_report": "backstop_data/ci_report"
},
"report": ["browser"],
"engine": "chrome",
"engineFlags": [],
"asyncCaptureLimit": 5,
"asyncCompareLimit": 50,
"debug": false,
"debugWindow": false
}
Default Configuration
{
"label": "BackstopJS Homepage",
"cookiePath": "backstop_data/engine_scripts/cookies.json",
"url": "https://garris.github.io/BackstopJS/",
"referenceUrl": "",
"readyEvent": "",
"readySelector": "",
"delay": 0,
"hideSelectors": [],
"removeSelectors": [],
"hoverSelector": "",
"clickSelector": "",
"postInteractionWait": "",
"selectors": [],
"selectorExpansion": true,
"misMatchThreshold" : 0.1,
"requireSameDimensions": true
}
14
On the CIPD PM build we have incorporated it in to the HTML dev process.
When components are created a new test scenario is added to the backstop
tests.
This captures the components from the locally setup demo site.
This creates a reference image for that component that is then checked in to
source control.
Project Setup
What to Test
15
16
Visual regression testing is the process of creating reference images for each
component as they are created. This allows a comparison over time to monitor
changes.
Ran by the developer on their local development environment after changes
made to ensure no regressions issues have occurred due to their changes.
Why is that useful?
• This allows the developer greater test coverage than they could do manually
when ensuring their changes have not had regression impact on other
components.
• Allows greater detail comparison than possible reviewing the site manually.
E.g. If the font size is increased by 1px then it will be caught which could
easily be missed.
• Allows developers without a full understanding of the project have a safety
net about what the site should look like before their changes.
Visual Regression Testing
17
Using visual testing to compare different environments involves comparing one
site against another . The reference images are not stored beyond the test.
A reference image set is generated form one site and then another site is
compared against them
Why is that useful?
• To ensure that the HTML created has been integrated within the website
correctly and is rendering out in the same way. This highlights mistakes with
the integration or conflicting integrations that are not present on the HTML
Demo site.
• Running regression tests against the Staging and Live instances of a site to
see where the changes have occurred.
• When setting up comparison between a demo and dev site powered by a
CMS this can make the tests brittle so the tests pages need to be isolated
from changes in content as possible.
Environment Comparison
18
Within a site their are hover states and interactive components. These need the
mouse or user interactions to trigger them.
How to test these in a headless browser?
Backstop allows scripts to be ran before a screenshot is captured. Using the
chromy API you can move the mouse and manipulate the page through script.
This allows testing of :
• Field entry
• Hover states
• Interactive items
• Video loading – Playing
• Ajax content loading
Chromy Git Hub URL : https://github.com/OnetapInc/chromy
Interacting with the site
19
When capturing items it is much quicker to capture components on a page than
the whole page to compare them
• It is much quicker to compare the smaller images
• It is more relevant the items that are being tested.
This allows the components themselves to be bested in isolation and most of our
development work is done using component based development.
Smaller is better
Setup in your project
20
Process to include in you
project
21
Install
Backstop
init within you
project Create
scenarios
Run Baseline
generation
Keep running
and
monitoring
22
Prerequisites
• Node – (If you don’t already have it)
• Chrome - (61 needed but latest is 62.0.3202.62 at the moment)
• Update node to v8+
• Update npm to v5+
On Windows you will also need to have python installed
> PhantomJS needs Python -- please make sure you have Python installed...
see https://github.com/garris/BackstopJS/issues/185
Installation
• npm install -g backstopjs
• Installs backstop globally
This allows the backstop command to be ran from anywhere on the PC.
Install Backstop - Globally
23
Backstop can also be installed as a grunt dependency and called through grunt.
var backstopjs = require('backstopjs');
grunt.registerTask('backstop', 'BackstopJS integration', function(cmd) {
// cmd is either 'reference', 'test', or 'openReport'
var done = this.async();
backstopjs(cmd).then(function() {
done(true);
}).catch(function() {
done(false);
});
});
This allows it to be integrated in to the existing grunt tasks.
Including it in grunt also allows the dynamic creation of the scenarios using
variables instead of them being held in a JSON file.
Installing Backtop in Grunt
24
There is a basic backstop.json file that is created when you run backstop init.
This can be edited to add in your own scenarios.
This is an example of the setup we are using.
Key Points:
• The Viewports are defined for use in all tests
• resembleOutputOptions define the comparison colours and display style
• The asyncCaptureLimit and asyncCompareLimit are used to control the
number of threads and memory the tests use.
• "engine": "chrome“ defines that the chrome headless browser will be used.
Configuration Setup
{
"id": "pm",
"viewports": [
{
"label": "phone",
"width": 320,
"height": 1024
},
{
"label": "tablet",
"width": 768,
"height": 1024
},
{
"label": "desktop",
"width": 1268,
"height": 1024
}
],
"onBeforeScript": "chromy/onBefore.js",
"onReadyScript": "chromy/onReady.js",
"scenarios": [
{
……..
}
],
"resembleOutputOptions": {
"errorColor": {
"red": 255,
"green": 0,
"blue": 255
},
"errorType": "movement",
"transparency": 0.3,
"ignoreAntialiasing" : true
},
"asyncCaptureLimit": 25,
"asyncCompareLimit": 400,
"engineFlags": [],
"engine": "chrome",
"report": ["browser","CI"],
"debug": false,
"debugWindow": false
}
25
For scenarios setup there are two key parameters :
• url
• selectors
These define the page the what will be captured on the page. The selectors
used simple CSS selectors to identify elements on the page to capture.
To capture the contents of the <div id=“header”> then the #header is used as a
selector.
If a referenceUrl is specified then this will be used instead of the url when
generating the reference images. This is used when comparing sites.
The hideSelectors parameter is used to define the elements on the page that
should not be shown. This is useful to hide cookie notifications for example that
could obscure capture of components.
Scenario Setup
{
"label": "header",
"url": "http://people-management.local/bb-prototype-
website_example_grid.html",
"selectors": [
"#header"
],
"hideSelectors": [
"#monitor_mq",
".grid",
".visible-grid-btn"
],
"readySelector": ".page-loaded“
},
{
"label": "podcast",
"url": "http://beta.pmdev.co.uk/test/podcast",
"referenceUrl": "http://internal.people-
management.demo.building-blocks.com/bb-prototype-
website_components-podcast.html",
"selectors": [
".podcast-container"
],
"hideSelectors": [
"#monitor_mq",
"#SEStartButton",
"#cookieMsgWrapper2",
".grid",
".visible-grid-btn"
]
}
26
Scenario Setup :
"onReadyScript": "burgerMenuTest.js",
Chromy Script :
module.exports = function (chromy, scenario) {
var clickSelector = scenario.clickSelector;
var postInteractionWait = scenario.postInteractionWait; // selector [str] | ms [int]
if (clickSelector) {
chromy
.wait(clickSelector)
.click(clickSelector)
}
if (postInteractionWait) {
chromy.wait(postInteractionWait);
}
};
Chromy
Backstop JS the future …..
27
• Catch Regression
• Allow comparison between DEMO HTML and Website
• Quick to setup
• Easily integrate in to existing HTML build process

More Related Content

PPTX
BackStopJS - how to avoid visual regression of our web application?
PDF
UI Testing Automation
PDF
Playwright: A New Test Automation Framework for the Modern Web
PDF
What's new in selenium 4
PPTX
Browser Automation with Playwright – for integration, RPA, UI testing and mor...
PPTX
Automation test framework with cucumber – BDD
PPTX
Automation - web testing with selenium
PPTX
Software testing
BackStopJS - how to avoid visual regression of our web application?
UI Testing Automation
Playwright: A New Test Automation Framework for the Modern Web
What's new in selenium 4
Browser Automation with Playwright – for integration, RPA, UI testing and mor...
Automation test framework with cucumber – BDD
Automation - web testing with selenium
Software testing

What's hot (20)

PPTX
BDD WITH CUCUMBER AND JAVA
PDF
What is Software Testing | Edureka
PPT
Test automation process
PPTX
Automated UI Testing
PPTX
Selenium with java
PPTX
WHITE BOX TESTING ashu.pptx
PPTX
Writing automation tests with python selenium behave pageobjects
PPTX
Cucumber With Selenium
PPT
Test Automation Framework Designs
PPTX
How to Design a Successful Test Automation Strategy
PPT
Software Testing
PPTX
Introduction to Selenium Web Driver
PPTX
Selenium
PDF
[Infographic] Challenges in Test Automation
PPT
PPTX
Visual regression with applitools eyes
PPTX
Integrate OutSystems With Office 365
PPTX
Maven
PDF
Automated Testing
PDF
Getting started with appium
BDD WITH CUCUMBER AND JAVA
What is Software Testing | Edureka
Test automation process
Automated UI Testing
Selenium with java
WHITE BOX TESTING ashu.pptx
Writing automation tests with python selenium behave pageobjects
Cucumber With Selenium
Test Automation Framework Designs
How to Design a Successful Test Automation Strategy
Software Testing
Introduction to Selenium Web Driver
Selenium
[Infographic] Challenges in Test Automation
Visual regression with applitools eyes
Integrate OutSystems With Office 365
Maven
Automated Testing
Getting started with appium
Ad

Similar to Technical Tips: Visual Regression Testing and Environment Comparison with Backstop.JS (20)

PDF
Automated Visual Regression Testing by Dave Sadlon
PPTX
Visual Regression Testing: In search of an Ember solution
PPTX
Automate testing with behat, selenium, phantom js and nightwatch.js (5)
PPTX
Colorful world-of-visual-automation-testing-latest
PDF
WebPagetest Power Users - Velocity 2014
PPTX
Visual regression testing
PPTX
Amplexor - Drupal Camp Romania 2015 - Front-end testing
PDF
BrowserArchitecture_ClientSide.pdf
PPTX
Testing CSS - Front end ops by Arpit Maheshwari
PPTX
BrowserArchitecture_ClientSide.pptx
PPTX
Testing Single Page Webapp
PPTX
Slow, Flaky and Legacy Tests: FTFY - Our New Testing Strategy at Net-A-Porter...
ODP
Web QA Gaia/B2G/Firefox OS front-end automation
PPT
How to do better Quality Assurance for Cross-Browser Testing
PDF
Headless Browser – A Stepping Stone Towards Developing Smarter Web Applicatio...
PDF
11 Top Cross Browser Testing Tools to Know About.pdf
PDF
11 Top Cross Browser Testing Tools to Know About.pdf
PDF
Ultimate Guide to Cross Browser Testing
PPTX
Edge of the Web
Automated Visual Regression Testing by Dave Sadlon
Visual Regression Testing: In search of an Ember solution
Automate testing with behat, selenium, phantom js and nightwatch.js (5)
Colorful world-of-visual-automation-testing-latest
WebPagetest Power Users - Velocity 2014
Visual regression testing
Amplexor - Drupal Camp Romania 2015 - Front-end testing
BrowserArchitecture_ClientSide.pdf
Testing CSS - Front end ops by Arpit Maheshwari
BrowserArchitecture_ClientSide.pptx
Testing Single Page Webapp
Slow, Flaky and Legacy Tests: FTFY - Our New Testing Strategy at Net-A-Porter...
Web QA Gaia/B2G/Firefox OS front-end automation
How to do better Quality Assurance for Cross-Browser Testing
Headless Browser – A Stepping Stone Towards Developing Smarter Web Applicatio...
11 Top Cross Browser Testing Tools to Know About.pdf
11 Top Cross Browser Testing Tools to Know About.pdf
Ultimate Guide to Cross Browser Testing
Edge of the Web
Ad

More from Building Blocks (8)

PPTX
Transforming to a new Digital DNA
PDF
Digital Foundations: 3 Simple Steps to Changing your Digital DNA
PPTX
How to create a compelling business case for Digital Transformation
PPTX
Transforming One Small Step At A Time: Optimisation & Testing
PPTX
Tackling 8 Digital Challenges of the Agrochemical, Industrial & Specialty Che...
PPTX
The importance of testing in a web design & build project
PDF
Going Digital First with Publishing for Lloyd's Register
PDF
Accelerating Eurocamp's Digital Transformation
Transforming to a new Digital DNA
Digital Foundations: 3 Simple Steps to Changing your Digital DNA
How to create a compelling business case for Digital Transformation
Transforming One Small Step At A Time: Optimisation & Testing
Tackling 8 Digital Challenges of the Agrochemical, Industrial & Specialty Che...
The importance of testing in a web design & build project
Going Digital First with Publishing for Lloyd's Register
Accelerating Eurocamp's Digital Transformation

Recently uploaded (20)

PDF
Machine learning based COVID-19 study performance prediction
PDF
Shreyas Phanse Resume: Experienced Backend Engineer | Java • Spring Boot • Ka...
PPTX
Understanding_Digital_Forensics_Presentation.pptx
PDF
Electronic commerce courselecture one. Pdf
PDF
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
PDF
Mobile App Security Testing_ A Comprehensive Guide.pdf
PDF
Agricultural_Statistics_at_a_Glance_2022_0.pdf
PDF
How UI/UX Design Impacts User Retention in Mobile Apps.pdf
PDF
Spectral efficient network and resource selection model in 5G networks
PPTX
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
PDF
KodekX | Application Modernization Development
PPT
“AI and Expert System Decision Support & Business Intelligence Systems”
PPTX
Detection-First SIEM: Rule Types, Dashboards, and Threat-Informed Strategy
PPT
Teaching material agriculture food technology
PPTX
MYSQL Presentation for SQL database connectivity
PDF
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
PPTX
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
PDF
Empathic Computing: Creating Shared Understanding
PPTX
20250228 LYD VKU AI Blended-Learning.pptx
PDF
Chapter 3 Spatial Domain Image Processing.pdf
Machine learning based COVID-19 study performance prediction
Shreyas Phanse Resume: Experienced Backend Engineer | Java • Spring Boot • Ka...
Understanding_Digital_Forensics_Presentation.pptx
Electronic commerce courselecture one. Pdf
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
Mobile App Security Testing_ A Comprehensive Guide.pdf
Agricultural_Statistics_at_a_Glance_2022_0.pdf
How UI/UX Design Impacts User Retention in Mobile Apps.pdf
Spectral efficient network and resource selection model in 5G networks
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
KodekX | Application Modernization Development
“AI and Expert System Decision Support & Business Intelligence Systems”
Detection-First SIEM: Rule Types, Dashboards, and Threat-Informed Strategy
Teaching material agriculture food technology
MYSQL Presentation for SQL database connectivity
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
Empathic Computing: Creating Shared Understanding
20250228 LYD VKU AI Blended-Learning.pptx
Chapter 3 Spatial Domain Image Processing.pdf

Technical Tips: Visual Regression Testing and Environment Comparison with Backstop.JS

  • 2. 1 I would define visual testing as using a headless browser to capture and compare images of components of a website. This can be used to implement unit tests for how the CSS is rendering the HTML in a browser. Not intended to test how the HTML is generated or replace Unit tests in a back end service layer. Not intended to replace cross browser testing. Allows comparison between sites or comparing against a reference image captured form the same site before changes were made. What do you mean by Visual Testing ?
  • 3. 1. Chrome Headless 2. Setup and Examples 3. What to test 4. Setup in your project Contents :
  • 4. Chrome Headless 3 What has changed ? Why is this a big deal ?
  • 5. 4  Headless Chrome is shipping in Chrome 59. It's a way to run the Chrome browser in a headless environment. Essentially, running Chrome without chrome! It brings all modern web platform features provided by Chromium and the Blink rendering engine to the command line. Why is that useful? A headless browser is a great tool for automated testing and server environments where you don't need a visible UI shell. For example, you may want to run some tests against a real web page, create a PDF of it, or just inspect how the browser renders an URL. Announcement blog : https://developers.google.com/web/updates/2017/04/headless-chrome How to Get it: As long as you have a version of chrome installed greater than Chrome 59 – ( Chrome 61 on Windows) you already have it  Chrome Headless
  • 6. 5  BackstopJS automates visual regression testing of your responsive web UI by comparing DOM screenshots over time. Backstop is a open source project to run visual tests using headless browsers to capture screenshots. Originally ran using either PhantomJS or SlimerJS headless browser libraries. These were implementations of WebKit (Chrome) or Gecko (Firefox). From release v3.0.19 this now supports Chrome headless. The latest release has a lot of stability improvements v3.0.27 Git Hub URL : https://github.com/garris/BackstopJS Features : • Allows capture and comparison of screenshots form headless browser • Excellent comparison tool to identify and highlight differences • Allows setup of multiple breakpoints to test responsive sites • Uses simple CSS selectors to identify what to capture • Pre capture scripts can be enabled to be ran Backstop JS
  • 7. 6 Chrome is based on WebKit but using PhantomJS as a headless browser caused issues making it unusable. There would be styling issues that would be thrown that would not be present on the site when viewed through chrome. As visual testing is about how the site renders this causes the tests to be unreliable. When I investigated implementing visual testing this was the stumbling block each time. Chrome != Webkit
  • 9. 8 • backstop init : Set up a new BackstopJS instance -- specify URLs, cookies, screen sizes, DOM selectors, interactions etc. (see examples directory) • backstop test : BackstopJS creates a set of test screenshots and compares them with your reference screenshots. Any changes show up in a visual report. (Run this after making CSS changes as many times as needed.) • backstop approve : If the test you ran looks good, then go ahead and approve it. Approving changes will update your reference files with the results from your last test. Future tests are compared against your most recent approved test screenshots. The BackstopJS workflow
  • 14. 13 { "id": "backstop_default", "viewports": [ { "label": "phone", "width": 320, "height": 480 }, { "label": "tablet", "width": 1024, "height": 768 } ], "onBeforeScript": "chromy/onBefore.js", "onReadyScript": "chromy/onReady.js", "scenarios": [ ………… ], "paths": { "bitmaps_reference": "backstop_data/bitmaps_reference", "bitmaps_test": "backstop_data/bitmaps_test", "engine_scripts": "backstop_data/engine_scripts", "html_report": "backstop_data/html_report", "ci_report": "backstop_data/ci_report" }, "report": ["browser"], "engine": "chrome", "engineFlags": [], "asyncCaptureLimit": 5, "asyncCompareLimit": 50, "debug": false, "debugWindow": false } Default Configuration { "label": "BackstopJS Homepage", "cookiePath": "backstop_data/engine_scripts/cookies.json", "url": "https://garris.github.io/BackstopJS/", "referenceUrl": "", "readyEvent": "", "readySelector": "", "delay": 0, "hideSelectors": [], "removeSelectors": [], "hoverSelector": "", "clickSelector": "", "postInteractionWait": "", "selectors": [], "selectorExpansion": true, "misMatchThreshold" : 0.1, "requireSameDimensions": true }
  • 15. 14 On the CIPD PM build we have incorporated it in to the HTML dev process. When components are created a new test scenario is added to the backstop tests. This captures the components from the locally setup demo site. This creates a reference image for that component that is then checked in to source control. Project Setup
  • 17. 16 Visual regression testing is the process of creating reference images for each component as they are created. This allows a comparison over time to monitor changes. Ran by the developer on their local development environment after changes made to ensure no regressions issues have occurred due to their changes. Why is that useful? • This allows the developer greater test coverage than they could do manually when ensuring their changes have not had regression impact on other components. • Allows greater detail comparison than possible reviewing the site manually. E.g. If the font size is increased by 1px then it will be caught which could easily be missed. • Allows developers without a full understanding of the project have a safety net about what the site should look like before their changes. Visual Regression Testing
  • 18. 17 Using visual testing to compare different environments involves comparing one site against another . The reference images are not stored beyond the test. A reference image set is generated form one site and then another site is compared against them Why is that useful? • To ensure that the HTML created has been integrated within the website correctly and is rendering out in the same way. This highlights mistakes with the integration or conflicting integrations that are not present on the HTML Demo site. • Running regression tests against the Staging and Live instances of a site to see where the changes have occurred. • When setting up comparison between a demo and dev site powered by a CMS this can make the tests brittle so the tests pages need to be isolated from changes in content as possible. Environment Comparison
  • 19. 18 Within a site their are hover states and interactive components. These need the mouse or user interactions to trigger them. How to test these in a headless browser? Backstop allows scripts to be ran before a screenshot is captured. Using the chromy API you can move the mouse and manipulate the page through script. This allows testing of : • Field entry • Hover states • Interactive items • Video loading – Playing • Ajax content loading Chromy Git Hub URL : https://github.com/OnetapInc/chromy Interacting with the site
  • 20. 19 When capturing items it is much quicker to capture components on a page than the whole page to compare them • It is much quicker to compare the smaller images • It is more relevant the items that are being tested. This allows the components themselves to be bested in isolation and most of our development work is done using component based development. Smaller is better
  • 21. Setup in your project 20
  • 22. Process to include in you project 21 Install Backstop init within you project Create scenarios Run Baseline generation Keep running and monitoring
  • 23. 22 Prerequisites • Node – (If you don’t already have it) • Chrome - (61 needed but latest is 62.0.3202.62 at the moment) • Update node to v8+ • Update npm to v5+ On Windows you will also need to have python installed > PhantomJS needs Python -- please make sure you have Python installed... see https://github.com/garris/BackstopJS/issues/185 Installation • npm install -g backstopjs • Installs backstop globally This allows the backstop command to be ran from anywhere on the PC. Install Backstop - Globally
  • 24. 23 Backstop can also be installed as a grunt dependency and called through grunt. var backstopjs = require('backstopjs'); grunt.registerTask('backstop', 'BackstopJS integration', function(cmd) { // cmd is either 'reference', 'test', or 'openReport' var done = this.async(); backstopjs(cmd).then(function() { done(true); }).catch(function() { done(false); }); }); This allows it to be integrated in to the existing grunt tasks. Including it in grunt also allows the dynamic creation of the scenarios using variables instead of them being held in a JSON file. Installing Backtop in Grunt
  • 25. 24 There is a basic backstop.json file that is created when you run backstop init. This can be edited to add in your own scenarios. This is an example of the setup we are using. Key Points: • The Viewports are defined for use in all tests • resembleOutputOptions define the comparison colours and display style • The asyncCaptureLimit and asyncCompareLimit are used to control the number of threads and memory the tests use. • "engine": "chrome“ defines that the chrome headless browser will be used. Configuration Setup { "id": "pm", "viewports": [ { "label": "phone", "width": 320, "height": 1024 }, { "label": "tablet", "width": 768, "height": 1024 }, { "label": "desktop", "width": 1268, "height": 1024 } ], "onBeforeScript": "chromy/onBefore.js", "onReadyScript": "chromy/onReady.js", "scenarios": [ { …….. } ], "resembleOutputOptions": { "errorColor": { "red": 255, "green": 0, "blue": 255 }, "errorType": "movement", "transparency": 0.3, "ignoreAntialiasing" : true }, "asyncCaptureLimit": 25, "asyncCompareLimit": 400, "engineFlags": [], "engine": "chrome", "report": ["browser","CI"], "debug": false, "debugWindow": false }
  • 26. 25 For scenarios setup there are two key parameters : • url • selectors These define the page the what will be captured on the page. The selectors used simple CSS selectors to identify elements on the page to capture. To capture the contents of the <div id=“header”> then the #header is used as a selector. If a referenceUrl is specified then this will be used instead of the url when generating the reference images. This is used when comparing sites. The hideSelectors parameter is used to define the elements on the page that should not be shown. This is useful to hide cookie notifications for example that could obscure capture of components. Scenario Setup { "label": "header", "url": "http://people-management.local/bb-prototype- website_example_grid.html", "selectors": [ "#header" ], "hideSelectors": [ "#monitor_mq", ".grid", ".visible-grid-btn" ], "readySelector": ".page-loaded“ }, { "label": "podcast", "url": "http://beta.pmdev.co.uk/test/podcast", "referenceUrl": "http://internal.people- management.demo.building-blocks.com/bb-prototype- website_components-podcast.html", "selectors": [ ".podcast-container" ], "hideSelectors": [ "#monitor_mq", "#SEStartButton", "#cookieMsgWrapper2", ".grid", ".visible-grid-btn" ] }
  • 27. 26 Scenario Setup : "onReadyScript": "burgerMenuTest.js", Chromy Script : module.exports = function (chromy, scenario) { var clickSelector = scenario.clickSelector; var postInteractionWait = scenario.postInteractionWait; // selector [str] | ms [int] if (clickSelector) { chromy .wait(clickSelector) .click(clickSelector) } if (postInteractionWait) { chromy.wait(postInteractionWait); } }; Chromy
  • 28. Backstop JS the future ….. 27 • Catch Regression • Allow comparison between DEMO HTML and Website • Quick to setup • Easily integrate in to existing HTML build process