Selenium and Open Source Advanced Testing

Selenium at Scale: Advanced tips for success in open source testing in 2017 Web: perfectomobile.com Twitter: @perfectomobile
Selenium at Scale
Advanced tips for success in open source testing in 2017
Web: perfectomobile.com Twitter: @perfectomobile

What We’ll Cover
● From Selenium 2.0 to 3.0
● The Role of Open Source in Continuous Testing
● Quality Across Platforms: Mobile+Web
● General Q & A

Poll: Concerning Selenium updates...
● I have to upgrade more than one (my) machine
● We upgrade as soon as a new stable version is released
● After upgrade, some of our scripts are broken
● We don’t find all incompatibilities right after the upgrade
● We often significantly delay updating to avoid being beta testers

Our Presenters
David Dang
VP of Automation Solutions
Zenergy
Darrell Grainger
Quality Advocate /
Automation Expert
ThoughtWorks
Eran Kinsbruner
Lead Technical Researcher
Perfecto

Darrell Grainger
Quality Advocate / Automation Expert
ThoughtWorks
From Selenium 2.0 to 3.0
I
Selenium at Scale

A Brief History of Selenium
1.0
rich interface for browser operations
tweaks on every browser update
Support EoL a few years ago
2.0
building blocks to your own DSL
execute 1.0 and 2.0 script
Still supported*
2004 2007
Selenium RC (1.0) WebDriver
2016
Selenium 3.0
2012
Selenium
WebDriver
(2.0)

From Selenium 2.0 to 3.0
● No philosophical changes from 2.53.x to 3.0
● In theory, upgrading from 2.53 to 3.0 shouldn’t hard… in theory
● To open a Firefox browser in 2.53 using the Java bindings:
WebDriver driver = new Firefox();
● To open a Firefox browser in 3.0 using the Java bindings:
System.setProperty("webdriver.gecko.driver", ”~/geckodriver");
DesiredCapabilities dc = DesiredCapabilities.firefox();
WebDriver driver = new FirefoxDriver(dc);

What does Selenium 3.0 have?
Gain Loss
Java bindings Compatibility to 3.0 Hard stop for Java 6/7
Ruby bindings Compatibility to3.0 Need to use Ruby 2.x+
Deprecated functions New replacement functions Old functions removed
Safari 10+ Compatibility to 3.0 Does not work on Safari 9<
Firefox 47.0.1+ Compatibility to 3.0 Does not work on 47.0<
Support IE 9+ Compatibility to 3.0 IE8 not supported

Why stick with the devil you know (i.e. 2.0)?
● There is what the documentation says and what ACTUALLY works
● In certain tech stacks and websites, some things no longer work in 3.0
● Example: site which is using react.js and you need WebDriverWait calls
● However there are things which worked on Selenium 2.0 which no longer
work on Selenium 3.0.
○ For example, populating the From field on Southwest’s flight booking website.
○ This does not work when using Firefox
(example)

Code Example
driver.get("https://www.southwest.com/flight/?clk=GSUBNAV-AIR-BOOK");
WebElement fromInput = driver.findElement(By.cssSelector("#originAirport_displayed"));
waitFor.until(ExpectedConditions.elementToBeClickable(fromInput));
fromInput.click();
waitFor.until(ExpectedConditions.elementToBeClickable(fromInput));
fromInput.sendKeys("new york");
By resultsLocator = By.cssSelector("div.ac_results li");
waitFor.until(ExpectedConditions.numberOfElementsToBeMoreThan(resultsLocator, 3));
List<WebElement> suggestions = driver.findElements(resultsLocator);
for (WebElement suggestion : suggestions) {
String s = suggestion.getText();
System.out.println(s);
if(s.equals("New York/Newark, NJ - EWR")) {
suggestion.click(); // does not work with Firefox
break;
}
}
screenshot();

The usual Selenium baggage
● Environment setup is also a consideration
● Downloading the latest drivers and bindings
● Setting up nodes in a Selenium Grid
● Updating the development setup with the new drivers and bindings
● Will the developers need to upgrade the programming language?
● Will you have to upgrade the build agents on your CI?
● Do you need to have support and security vet the new technology?
● When working in enterprises, upgrading the tech stack can be a big deal
● You might need to involve different departments

When switching to Selenium 3.0...
● Selenium 3.0 basically requires you to beta test the bindings
● Why not wait for other people to work out the bugs?
○ You need to support the latest browser
○ You need to use the latest binding languages
○ You’ll need to switch at some point, no better time than the present

More info on moving from Selenium 2.0 to 3.0
● Good place to find API documentation and changelogs:
○ http://www.seleniumhq.org/download/
● The chromedriver:
○ http://chromedriver.storage.googleapis.com/index.html
● The geckodriver for Firefox:
○ https://github.com/mozilla/geckodriver/releases
● The safari driver:
○ This is actually part of macOS
○ https://webkit.org/blog/6900/webdriver-support-in-safari-10/
● Selenium source code:
○ https://github.com/SeleniumHQ/selenium

Flash Q&A : Darrell
Quality Advocate / Automation Expert
ThoughtWorks

David Dang
Zenergy
The Role of Open Source
in Continuous Testing
II
Selenium at Scale

● As the industry standard tool for
browser automation, countless tools
are being created that extend or
integrate with Selenium WebDriver
● Tools like Appium and Protractor
extend WebDriver for use in other
technologies
● Tools like Cucumber integrate with
WebDriver so it can be used with
methodologies such as BDD
Application
Under Test
Selenium WebDriver
WebDriver: the foundation for open source testing

● More technical oriented compare purchase tools
● Decision on a test automation framework is not always straight forward
● Integration with existing toolset can be difficult
● Keeping up with the latest a greatest is always a struggle
● Limitation of technologies support
(client/server, ERP, CRM, mainframe, etc…)
Challenges with Open Source Testing at Scale

● What comes for free → WebDriver community, bindings, integrations
● What comes with free → costs: environment config, customization, skills
● What doesn’t come for free → People, technologies, process
○ Culture - Dev/Tests together?
○ Enterprise-ready support
○ Tools and external libraries
Key Considerations for Upgrading Selenium

Selenium 3.0: What you need to be successful
● What you need to be successful
○ Dedicated resources
○ Test automation strategy
○ Test automation framework (page object model, keyword, data-driven, etc…)
○ Integration with other tools/frameworks (Maven, TestNG, Apache POI, Log4J,
ReportNG, Serenity, Jenkins, etc…)
○ Development and maintainence plan

Flash Q&A : David
Zenergy

Eran Kinsbruner
Perfecto
Quality Across Platforms:
Mobile+Web
III
Selenium at Scale

Matching Tools to Requirements and Persona’s
● Different type of apps (Native/RWD/Web)
● Organizational skill set
● Context Based Testing (In-App/External)
● Embedded into workflow/IDE’s?
● Test coverage requirements
(Platforms/Capabilities)

A Day in A Life – Covering Real Environments Isn’t
Possible With Every Tool
12/3/2016
Automation ought to cover
the full user environment
not just the Application
Under Test!

Which Tool is the best fit for you?
12/3/2016
24
© 2016, Perfecto Mobile Ltd. All Rights Reserved.
SELENIUM APPIUM CALABASH
ESPRESSO XCTEST UI
When looking at today’s open-source mobile test automation landscape, there
are five highly-adopted test frameworks.
We have summarized each approach to help you decide which makes the most sense for you.
Each tool has advantages for your mobile and web testing depending on your needs.
While reading, consider which of the functions
you need most for your own test automation purposes.

Making the Decision
A Comparison
Detailed analysis of each test
automation tool:
The next slides offer a
breakdown of each tool

Selenium
• The best choice for web test automation
teams testing for responsive web design or
stand-alone web sites
• Less suitable for developing unit testing,
making this framework less appealing for
developers
• Its core test reports are not highly
informative and lack unique mobile related
insights

Appium
• Best suited for QA teams testing the
functionality of native, mobile web and
mobile hybrid apps across iOS and Android
• Less suitable for developers who wish to
develop and perform unit testing
• Appium reports are a bit limited from a
debugging and fast feedback loop
perspective, and do not include videos,
network logs and key vitals information

Calaba.sh
• Designed for organizations that work in
behavior-driven development workflows
• Offers an easy path to both develop and
test features in parallel, all in an easy
user-flow based language
• Appealing for both dev and QA practitioners
• Provides solid insights and reports to both
dev and QA teams

Espresso & XCTest UI
12/3/2016 © 2016, Perfecto Mobile Ltd. All Rights Reserved.
ESPRESSO
XCTEST UI
• Both are designed for target users
• Espresso is for Android and XCTest is for iOS
• Both tools are fully integrated into development IDEs such as Android
Studio/Xcode, and offer very easy-to-develop techniques, including test
recorders
• Fully maintained by Google and Apple, which assures they always support the
latest OS features so developers can stay ahead of the market and test
accordingly
• Support both unit testing types and functional UI testing
• Both are app-context only, which limits their abilities to test for user condition
scenarios
ESPRESSO

Things to Consider
• Lab: Having support for variety of frameworks is important for enterprises
• Flexibility
• Autonomy
• complete E2E coverage
• unattended testing
• Automation: support for multiple OSS frameworks becomes important
• Different “persona’s” need different tools under 1 lab
• Cross platform scripting
• Robust framework to support high test coverage
• Analysis: Digital reporting is necessary for teams to take action and fix
issues faster & earlier
When choosing a test automation tool

LIVE DEMO – RWD Cross Platform Automation

The Digital Solution
“One Script, One Lab”
for all digital platforms
Digital Test Suite Reporting for Agile
Teams
One Script
For All Digital Platforms
One Lab – Web & Mobile
One Secure Cloud
Manual & Automated Testing

Flash Q&A : Eran
Perfecto

Thank you!
Next webinar: projectquantum.io
bit.ly/perfecto-bdd-quantum

Selenium and Open Source Advanced Testing

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to Selenium and Open Source Advanced Testing (20)

Selenium and Open Source Advanced Testing