Android e2e testing at mercari

マスタータイトルの書式設定
• マスターテキストの書式設定
Vishal
Android Test Night #1
Android e2e Testing at Mercari

• マスターテキストの書式設定 Software Engineer in Test
Case Study (Mobile Automation)
SWET (past)
Core Member
Vishal Banthia @vbanthia @vbanthia_

Agenda
• About Mercari US Android App
• Android UITest Architecture & Design
• Continuous Integration

In June, we had a major update in mercari US app.
mercari US app
Native Native Native React-Native
Not Just UI Changes, but complete architecture shift

• マスターテキストの書式設定• While deciding new architecture, easy testability was always a priority
• One of the reason to use dependency-injection pattern is above and
we use Dagger2 for this. With this, mocking objects is pretty easy and
can increase UnitTest coverage
• We always try to keep low-level UnitTest coverage high following Test
Pyramid Principle
New Architecture
Note: More technical details can be read here (http://tech.mercari.com/entry/2017/07/19/123443)

But UITesting is always challenging
and was so for US

• After joining mercari, this was my first big project. Following objectives
were in mind:
• Adding new test scenarios and maintaining old ones should be painless
• High reliability so that flaky tests are not wasting developers and QA
engineers time
• Scalable architecture to minimize test run time
• Tests should run on real devices instead of emulators
• Minimum operational cost for infrastructure and CI management
• Test reports should have all the important information required for
debugging if test fails
• Super portable, should be easy to copy for other products
Android UITesting

UITest High Level Architecture
Ruby
RSpec
AppiumKit
Appium
UIAutomator2
Test Script Automation tool App Under Test
(MercariKit)

appium_kit (Ruby Library)
A simple ruby gem which extends famous selenium testing
ruby library such as Capybara, SitePrism for Appium

Selenium Web Testing in Ruby
selenium-webdriver
capybara watir
site_prism page-object
rspec cucumber test-unit

Quick Overview
● selenium-webdriver is the ruby binding for Selenium
● capybara provides DSL to help writing web tests easily
● page-object design pattern helps in maintaining test easily
● site_prism provides DSL to write pages easily
These libraries have proved themselves for so long, we wanted to use something which we can
rely and can also use same stack for web-testing that is why, simple appium_kit gem.

appium_kit
selenium-webdriver
capybara
site_prism
rspec
appium-driver
capybara-appium
site_prism
rspec helpers / formatter
appium_kit

UITest Low Level Architecture
user_profile_spec.rb (rspec)
UserProfilePage < SitePrism::PageUtils
Helpers

Overview
• Pages
• All the application related information such as ids, static text etc are tied
with Page Object. Test will talk to application using either helpers or
pages. Test can not access application directly.
• 1 Activity ~= 1 Page
• Helpers
• Helper binds pages together
• Utils
• Utilities such as RandomGen, AdbUtil etc. Not directly related to AUT
• Configs
• Static testdata

Running Tests

Rake Tasks
bundle exec download_apk bundle exec rake spec
Environment Variables
● BUILD_NUMBER
Environment Variables
● ENVIRONMENT
● FEATURES
● ...

Dockerizing UITests
• Dockerizing tests helps in sandboxing environment.
• Deploying / Running tests becomes very easy. Do not
need to worry about run environment
• Many more benefits …
More Details: https://github.com/vbanthia/appium-docker-demo

Dockerfile
https://github.com/vbanthia/docker-appium

run_uitest.sh
• Reserve a device on STF
• Remote connect
• Download apk
• Start appium server
• Run test

Continuous Integration

Mercari Android CI (Tools & Service)
• Bitrise: To build application
• Google Cloud Storage: To store built apks
• Google Container Registry: To store docker images
• OpenSTF: For Android Device Farm
• CircleCI 2.0: Build docker image & run tests in parallel containers
• CircleCI Artifacts: To store test reports
• AWS Lambda: To trigger nightly builds
• Slack: Notification & manual triggers

Application Build Pipeline
Developer Github Bitrise GCS
1. Developer pushes code to Github
2. Github triggers Bitrise
3. Bitrise build app and upload it to GCS
This make sure that we have all the apks
in GCS with their build number linked

UITest docker image build pipeline
Developer Github CircleCI GCR
1. Developer pushes code to Github
2. Github triggers CircleCI
3. CircleCI build docker image for uitest
and pushed it to google container
registry
This make sure that we have all the
docker image of latest test code any time

UITest Pipeline

OpenSTF

UITests Pipeline
CircleCI Job
AWS Lambda
Slack
Connect remote deviceDownload apk
Run UITests
(device1)
Run UITests
(device2)
Run UITests
(device3)
Test reports
CircleCI Artifact

Benefits
• Everything is on cloud, we only have devices connected to
mac-mini which only needs adb to work
• Always ready to run
• No need to worry about on-premise infra (Jenkins etc...)

Test Reports

RSpec HTML Formatter2
● https://github.com/vbanthia/rspec_html_formatter2
● Features:
o Beautiful html report using Bootstrap css/js
o Supports screenshots & screenrecords

Thank you

Android e2e testing at mercari

More Related Content

What's hot (20)

Similar to Android e2e testing at mercari (20)

More from Vishal Banthia (6)

Android e2e testing at mercari