Automate test, tools, advantages, and disadvantages

Automate Test, Tools, advantages,
and disadvantages
Majid Hosseini
Scala Developer @ ModernPal
www.linkedin.com/in/majidin

Agenda
● Introduction to test and
automation
● Test Engineer & QA Role
● Introduction to some test
automation tools
● Contract Testing
A QA engineer walks into a bar, and orders a beer
Then he orders 0 beers.
Then he orders 999999999999 beers.
Then he orders an aardvark.
Then he orders nothing.
Then he orders -1 beers.
Then he orders NULL beers.
Then he orders asnwikfjsdf.
Then he orders a
"><script>give_me_your_credit_card()</script>.
Finally, the QA engineer leaves without paying, comes back,
and asks for the tab.

More and more test....
Unit Test
Integration Test
end to end Test
e-2-e Test
UI Test
Acceptance Test
WhiteBox Test
BlackBox Test
Contract Test
Performance Test
Manual Test
GUI Test
High-level Test
Smoke Test
Load Test
Stress Test
Component Test
User Acceptance Test
Layout Test
Beta Test
Installation Test
Compatibility Test
Alpha Test
Regression Test
Functional Test
Non-functional Test
Destructive Test
Accessibility Test
Security Test
Ad-hoc Test
Agile Test
API Test
Backward compatibility Test
Branch Test
Exploratory Test
Fuzz Test
GlassBox Test
Gorilla Test!
Golden path testing
Interface Test
Happy path testing
Internationalization Test
Penetration Test Sanity Test
Scalability Test
Stability Test
System Test
Soak Test
Volume Testing
Vulnerability Test
Recovery Test

Unit Test:
“A Stack” should "throw NoSuchElementException if an empty stack is
popped" in {
val emptyStack = new Stack[String]
assertThrows[NoSuchElementException] {
emptyStack.pop()
}
}
Integration Test:
"Steering repository " should "find steerings by their ID" in {
Await.result(repository.add(testSteering), Duration.Inf)
Await.result(repository.add(testSteeringDifferentId), Duration.Inf)
val steering = Await.result(repository.findByKey(testSteeringId), Duration.Inf)
assert(steering.isDefined)
assert(steering.get.id === testSteering.id)
}
Test Level
Unit Test
Integration Test
High Level Test
High level Test:
"POST /steering_points" should
"Return 400 Badrequest error if name is empty string" in {
val req = baseSteeringPoint ++ Json.obj("name" -> "")
val response = Await.result(steeringPointsUrl.post(req), 5 second)
val result = Json.parse(response.body).asOpt[ErrorDTO]
assert(response.status == 400)
assert(result.isDefined)
}

High Level Test
• Simulate real user scenarios
• Verify the entire flow of the application
• Increase test coverage

Unit Test
● Fast
● Reliable
● Isolation Failures
● Easy Maintain

Test Pyramid
Keep in mind that all testing should be done
on the lowest level possible since
every step up the pyramid
makes tests more complex
and error prone.

Test for CI and CD (Production Pipeline)

Uncertainty in Test (Martin Fowler)
●
●
●
●
●

QA and Test Roles in Companies
● Separated “QA & Tester” Role
● No individual “QA & Tester” Role but SRE, DevOps and GOOD Product &
Development teams...

● Test & QA Designer
○ Comprehensive knowledge about the product and business domain
● Test Executor/Manager
○ Good knowledge of product and business domain
○ High ability to plan and being creative
○ Strong ability in investigation
○ Strong ability in communication and reporting
● Test Engineer
○ Front-end TE
○ Backend TE
○ Mobile TE
○ Specific TE: Security, Performance , ..
● Site Reliability Engineering

1. Describe behavior in plain text (Test Scenarios)
2. Implement scenario steps in a language
3. Run the scenarios

few rules to get a good scenario
✓ Use a proper @Tag to group related scenarios & execute them together >>>
Jenkins works with @Tags.
✓ Use non-technical terms in your scenario.
✓ Use only 1 when in each scenario.
Gherkin - simple syntax:
@{Tags}
Given for preconditions/testdata
When for steps/actions
Then for expectations/checks
{Data input}

Cucumber
1. Describe behaviour (test scenario) in plain text (Gherkin)
@priceUpload
Scenario: I want to test if uploading a file works
Given a file with name “myfile.csv”
When I upload the file
Then email is received
And the status will change to “SUCCESSFUL” in “10” seconds
2. Write a step definition in a programming language

Frontend Testing (Web)
Selenium WebDriver
1. Identify an element e.g. “text field 1”
2. Perform some action e.g. type “hello!” and Submit
3. Verify
● What WebDriver is able to:
○ interactions on the page (e.g. fill, click, submit, isElementPresent)
○ interactions with web browser e.g. add parameters, refresh, navigate back
○ JavaScript execution
● What WebDriver is not designed for:
○ layout testing ("should be green/have a border/..." is possible, but not maintainable)
○ js console output

Mobile Test
Android iOS e2e
Espresso KIF
EarlGrey
Appium
Calabash

Crowd Testing
Example: www.test.io

Performance Testing
What are we looking for?
SLOs/SLAs
Example of KPIs:
• Response time
• Throughput
Several tools exist, depends on your needs:
1. JMeter
2. Gatling
3. Apache Benchmark

Contract Testing
The consumer defines what it expects from a specific request to a service
The provider and the consumer agree on this contract
The provider continuously verifies that the contract is fulfilled

Pact A Contract Testing Framework

Automate test, tools, advantages, and disadvantages

More Related Content

What's hot (20)

Similar to Automate test, tools, advantages, and disadvantages (20)

Recently uploaded (20)

Automate test, tools, advantages, and disadvantages