How to Identify Broken Links in Selenium WebDriver ?

Broken links are hyperlinks or URLs on a web page that are not functioning as expected, these links may be unreachable due to various reasons, such as missing resources, server errors, incorrect URLs. When a user clicks on a broken link, they  encounter error page or page is not reachable.

Selenium WebDriver identifies and handle these broken/dead links using HTTP request, A valid URL typically returns a 2xx status code, while broken links may have 4xx or 5xx status codes. 4xx codes indicate client-side errors, such as incorrect URLs or missing resources and 5xx codes indicate server response errors, which may occur if the server is down or misconfigured.

What causes a broken link?

• There may be various reason for dead links.
• Underlying pages are moved or deleted, or URL been changed.
• There could be typo errors, incorrect link syntax, incorrect format, moving to new domain ..
• Server-side errors, outage etc
• Third-party services with dependencies.

How do I track broken links?

• Testers finds way to identify these broken links based on their feasibility , however, below are some techniques testers identify.
• Manual inspections- it depends on complexity and size of application.
• There are many online broken link checkers, scanners, and tool such as W3C Link Checker, Dead Link Checker etc.
• Automated testing with Selenium WebDriver
• There are Monitoring tools such as Google Analytics or Google Search Console etc.

How to identify broken links in Selenium WebDriver?

You need to locate all hyperlinks , verify the valid and invalid using HTTP response code , refer these steps-

• Use Anchor <a> tag to fetch all hyperlinks on the web-page.
• Send HTTP request for the link
• Verify the HTTP response code for each link.
• Verify for valid or invalid https response , invalid response will be considered as 'broken links'
• Get all invalid links captured and print.

Refer these code snippet-

1. Get all links on the web page

List<WebElement> listoflinks = driver.findElements(By.tagName("a"));        

2. Validate URL present in the list, we will use getAttribute() function, Also we usee ‘href’ attributes to get valid url only from list .

for (WebElement el : listoflinks) {

			String url = el.getAttribute("href");
				URL urllinks = new URL(url);

}        

3. Send HTTP request-HttpURLConnection helps to create connection , also we use connect() method .

HttpURLConnection huc = (HttpURLConnection) urllinks.openConnection();
				huc.setRequestMethod("HEAD");
				huc.connect();        

4. Validate Links, we are using here a basic validation, user can apply validation based on their requirement.

resCode = huc.getResponseCode();
				if (resCode > 400) {

					System.out.println(url + "broken link");
					brokenLinkCount++;

				}        

Please refer to this code , talks about identifying broken links using Selenium WebDriver.

package Mvn_Selenium_Test1.Mvn_Selenium_Test1;

import java.net.HttpURLConnection;
import java.net.MalformedURLException;
import java.net.URL;
import java.util.List;
import java.util.concurrent.TimeUnit;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;

import io.github.bonigarcia.wdm.WebDriverManager;
public class BrokenLinks {
	public static void main(String[] args) {
		WebDriverManager.chromedriver().setup();
		WebDriver driver = new ChromeDriver();
		// maximize the window size
		driver.manage().window().maximize();
		// wait for 10 seconds
		driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
		// get the URL
		driver.get("https://www.google.com/");
		// finding hyperlinks
		List<WebElement> listoflinks = driver.findElements(By.tagName("a"));
		// get all URL
		int resCode = 200;
		int brokenLinkCount = 0;
		System.out.println("total count of link" + listoflinks.size());
		for (WebElement el : listoflinks) {
			String url = el.getAttribute("href");
			try {
				URL urllinks = new URL(url);
				HttpURLConnection huc = (HttpURLConnection) urllinks.openConnection();
				huc.setRequestMethod("HEAD");
				huc.connect();
				resCode = huc.getResponseCode();
				if (resCode > 400) {

					System.out.println(url + "broken link");
					brokenLinkCount++;
				}
			} catch (MalformedURLException e) {

			} catch (Exception e) {
			}
		}
		System.out.println();
	}
}