Visual Regression Testing Using Playwright.pdf

Visual Regression Testing Using
Playwright
Amid the fast-moving process of delivering software, the time spent on
reviewing web pages manually is progressively reducing. In order to
maintain the speed of modern development, we require tools that help
keep our designs and layouts preserved and safe from changes. That
is where Visual Regression Testing comes handy.
About Visual Regression Testing?
Visual regression testing focuses on image and pixel comparisons. It
involves identifying specific sections of a website and visually
inspecting them to detect any changes It works by comparing a
baseline image (or ‘master’ image) with screenshots captured during
subsequent tests

In other words, It checks that your website’s content, layout and
design stay the same after each change of code.
Example: For example, you have just changed the CSS of your
website by a small margin by making a small update . While in a
development environment a button seems perfect, in the live site, it is
a little off; the layout on mobile. This would have been identified by the
use of Visual Regression Testing.
Why Choose Playwright for Visual Regression
Testing?
Playwright is an excellent choice for visual regression testing because
it provides a powerful framework for automating browser interactions
while offering built-in tools for capturing screenshots and comparing
them. It allows you to:
● Check your tests in different browsers: Chromium, Firefox,
WebKit, on Windows, Linux and MacOS.
● Detects layout shifts, broken UI elements, and CSS issues.
● Run tests in parallel to speed up the process, supporting modern
development workflows.
Many modern developments require running of tests in parallel in
order to reduce the testing time. Playwright visual regression tool is
ready to use out of the box, and its compatibility with other testing
frameworks makes it the most convenient option for checking that the
UI of your application is correct at any given time on any platform with
any device.
Key Benefits of Visual Regression Testing with Playwright

1. Speed: When there are many releases, the time required for a
manual audit of each page of an application to detect visual
defects rapidly grows. Automated visual regression testing does
this, and all of it, in a very rapid manner.
2. Consistency: Since the visual checks are automated, they are
consistent. There’s no risk of human error where something is
overlooked during a manual review.
3. Cross-Browser Validation: Playwright supports cross-browser
visual testing or guarantees that the final product will function
and look the same in all potential environments.
4. Historical Insights: By tracking how the appearance of your site
evolves over time, you can identify what changes have been
made to your site’s appearance and what design decisions are
productive and which can lead to negative effects for the users.
Limitations of Visual Regression Testing
While powerful, visual regression testing does not cover every aspect
of application quality. Some areas it doesn’t cover include:
● Functional Testing: It won’t verify that features work correctly;
you’ll still need functional testing for that.
● Security Testing: Visual regression tests don’t assess
application security.
● Accessibility: Accessibility standards require separate testing
processes and are not covered by visual regression testing.
● Backend/API Testing: These are outside the scope of front-end
visual testing.

Key Methods in Playwright for Visual Regression
Testing
When working with Playwright for visual regression testing, two
primary assertions will help you compare the current state of your web
pages or components with reference snapshots:
1. .toHaveScreenshot()
This method is intended for comparing the current screenshot of a
webpage or some of its elements to the so-called ‘golden image
reflecting the state of the web page/element at the definite moment of
time when it was stored. As soon as it is launched, it takes a snapshot
of the current state of the page and then runs it against a base image.
Usage:
● It will be used when you need to provide a visual check on the
whole page or a given component.
● This is useful when trying to do comparison across the whole
page or just checking certain areas of the application for
alignment.
test('Visual comparison of page', async ({ page }) => {
await page.goto('https://test.com');
// Compare the full page screenshot
await expect(page).toHaveScreenshot();
});
How it works:
● During the first run, it produces a baseline image of the site. If
the comparison is carried out on subsequent runs, Playwright will
identify some changes and notify the user of the differences.

● You can pass additional options, such as animations: “disabled”
to avoid false positives caused by dynamic elements.
2. .toMatchSnapshot()
This method enables you to compare a string, Buffer, or any value; an
image, for example, or a JSON file, with a previously captured golden
or reference value. It is flexible and is not limited to the type of
comparisons that are basic to the facet models, where you can also
check other forms of data.
Usage:
● Use .toMatchSnapshot() when performing a specific data point
comparison, for example: HTML strings or component snapshots
and a reference value.
● In general it is applied to test single elements, components, or
API responses.
test('Compare element visual snapshot', async ({ page }) => {
await page.goto('https://test.com');
const screenshot = await page.screenshot();
// Compare the screenshot buffer with the reference image
expect(screenshot).toMatchSnapshot();
});
How it works:
● Similar to .toHaveScreenshot(), it takes a snapshot and in
consecutive runs, Playwright compares it to the current
screenshot or a data buffer to check for changes.
● This method is one of the more flexible around and can be used
for visual comparison of files, or for comparison of JSON
responses, the structure of HTML documents, or really any form
of serialized data.

Implementing visual testing using Playwright
Let’s implement visual regression testing using Playwright for the
website. I’ll provide you with code examples for:
● Visual testing of a single webpage.
● Visual testing of a specific element.
● Visual testing of the entire webpage.
Visual testing for a single web page
In this test, we will navigate and visually compare the entire page
against a baseline image. Playwright will then take a snapshot of the
page slider and check again in order to see whether something has
changed.
const { test, expect } = require('@playwright/test');
test('Visual comparison of the TestGrid.io homepage', async ({ page })
=> {
// Navigate to TestGrid.io homepage
await page.goto('https://testgrid.io/');
// Compare the entire page to a previously stored snapshot
await expect(page).toHaveScreenshot();
});
When we execute the above code it will fail the first time and a new
folder is created.
The folder tests/singlePage.spec.js is where your test files are located,
and baseline .png files are generated when you run visual regression
tests for the first time

The test-results folder is where you can find the Playwright visual
regression testing results. This folder will store the new screenshots
captured during the test run, along with any diff images that highlight
visual differences between the baseline and the current screenshots.
In the below screenshot, you can see the different created folders.
In the below screenshot, you can see the test case has failed and an
error has come.

Run the test case again, when we execute the test case second time it
will pass

Visual testing for a single element
In this case, we will target a specific element on the page, such as the
“Get Started” button. We will visually compare this button to ensure it
remains unchanged after each test run.
test('Visual comparison of the "Get Started" button', async ({ page })
=> {
await page.goto('https://testgrid.io/');
// Locate the "Get Started" button by its text or selector
const getStartedButton = page.locator('text=Get Started');
// Compare the "Get Started" button to the baseline image
await expect(getStartedButton).toHaveScreenshot();
});
When we execute the above code it fails as the baseline image does
not exist.

Lets execute the test case again this time as the baseline image exists
and compare the image with the baseline image. In the below
screenshot you can see only the image of particular element is
captured instead of whole page

Visual testing for full web page
Full-page visual comparison captures the entire height of a webpage,
which is useful for websites with scrollable content. However, the risk
of failure increases with larger areas.
To mitigate this, you can disable CSS animations (animations:
“disabled”) and set a tolerance for minor differences
(maxDiffPixelRatio: 0.2).
test('Full page visual comparison of TestGrid.io', async ({ page }) => {
await page.goto('https://testgrid.io/blog/');
// Capture the full page screenshot and compare with the baseline
await expect(page).toHaveScreenshot({ fullPage: true, animations:
"disabled", maxDiffPixelRatio: 0.2 });
});
When we execute the above code it fails as the baseline image does
not exist.

Lets execute the test case again this time as the baseline image exists
and compare the image with the baseline image. In the below
screenshot you can see full page screenshot is captured.
In the next section, you will see about Visual Regression Testing
using.

Visual Regression Testing in TestGrid
Visual Regression Testing ensures that an application’s appearance,
layout, and design remain consistent across various devices and
resolutions. It compares the visual output of the application against a
baseline, detecting any unintended changes to the user interface. it
provides an efficient and user-friendly approach to this process,
making it a valuable tool for maintaining a consistent and error-free UI.
Key Benefits of Using TestGrid for Visual Regression
Testing
Automated Baseline Creation:
The first time the test is run after adding the code, it takes a baseline
picture for itself and saves it. This appearance is used as a
benchmark against which subsequent test runs are made. Setup time
is little and this makes it easy to start with the visual testing without
having to manually create the baselines.
Simplified Comparison Process:
After executing the test, users can log into it, navigate to Automation
Sessions, and select the device on which the tests were run. The
platform displays timestamps for each executed test case, making it
simple to find and review results.
By clicking on the visual testing section, users can see the list of all
test cases, both passed and failed, providing a clear overview of the
test outcomes.
Detailed Insight into Failures:
If a test case fails, it enables users to click on the Compare button to
view differences between the baseline and current screenshots. Any
variations are highlighted with a red rectangle, allowing users to

quickly identify changes that may have impacted the visual
consistency of the application.
This detailed comparison feature helps pinpoint specific elements that
have shifted or altered, making it easier to address visual bugs.
Consistency Through Subsequent Executions:
For each subsequent test execution, it automatically compares the
newly captured screenshots against the previously established
baseline. This ensures that any new changes or issues are
immediately detected.
Users can run the same test case multiple times and track changes
over time, helping to ensure that updates or fixes do not introduce new
visual errors.
Real-Time Monitoring and Adjustment:
During execution, it allows users to observe the comparison process
in real-time. This real-time insight is valuable for identifying issues as
they occur and enables prompt adjustments.
In the next section you will see how we can execute the visual test
case in Test Grid
Execution and Result Analysis
Once the code is added, you can proceed with its execution. The
initial run will generate a baseline image, which will serve as the
reference for comparing images in future test runs. From the second
execution onwards, the visual testing results will be available for
review.
To see the execution results login into it, with valid credentials and
navigate to Automation Sessions.

Device Selection :
Select the device on which you have executed the test cases

Executed test cases :
You will see a list of timestamps for the executed test cases. Click on
the last timestamp of the test that you have executed.

Comparison :
Once the execution is finished, you will see the list of test steps with
the generated status, Click on visual testing tab which will display all
Pass and Failed test cases list.

You can see the reason for failure of the test case by clicking on the
Compare button to see the visual difference between 2 screenshots.
The difference will be marked by a red rectangle.
You can click on the Update Baseline button if you would like to set
the recent image as the baseline for the next test case execution.

Subsequent Executions:
From the second execution onwards, the screenshots will be
compared against the baseline. Run the same test case with the same
steps once again to see the visual testing results.
Conclusion
Visual regression testing with Playwright is a robust solution for
ensuring UI consistency and catching visual bugs early in the
development lifecycle. By automating this process, Playwright helps
you maintain the visual integrity of your web application across
multiple devices and browsers.
Source: This article was originally published at testgrid.io.

Visual Regression Testing Using Playwright.pdf

More Related Content

Similar to Visual Regression Testing Using Playwright.pdf (20)

More from Steve Wortham (20)

Recently uploaded (20)

Visual Regression Testing Using Playwright.pdf