[26]자동화, 계륵에 살 붙이기 : Evolution of Android Automation Test
2 likes1,459 views
자동화, 계륵에 살 붙이기 : Evolution of Android Automation Test 송의초경 N Tech Service/ Contents QA NTS Contents QA 팀에서 근무하고 있는 송의초경입니다. 오디오 플랫폼에서 주로 QA를 진행했고 네이버 뮤직을 거쳐 현재는 오디오 클립 QA를 담당하고 있습니다. 자동화 거버넌스 TF에서 자동화 라이브러리의 유지 보수도 함께 담당하고 있습니다.
![[26]자동화, 계륵에 살 붙이기 : Evolution of Android Automation Test](https://image.slidesharecdn.com/26automation-181102022301/85/26-Evolution-of-Android-Automation-Test-1-320.jpg)
![public class TestClass {
public AppiumDriver<WebElement> driver;
@BeforeClass
public void setup() throws Exception {
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability(MobileCapabilityType.PLATFORM_NAME, "Android");
capabilities.setCapability(MobileCapabilityType.UDID, "UDID");
capabilities.setCapability(MobileCapabilityType.PLATFORM_VERSION,"platform-version");
capabilities.setCapability(MobileCapabilityType.AUTOMATION_NAME, "UIAutomator2");
capabilities.setCapability(AndroidMobileCapabilityType.APP_PACKAGE, "package");
capabilities.setCapability(AndroidMobileCapabilityType.APP_ACTIVITY, "launchable-activity");
driver = new AppiumDriver<WebElement>(new URL("http://127.0.0.1:4723/wd/hub"), capabilities);
driver.manage().timeouts().implicitlyWait(15, TimeUnit.SECONDS);
}
@Test
public void scenario_01() {
driver.findElement(By.id("tv_home_weekly_schedule_title")).click();
}
@Test
public void scenario_02() {
driver.findElement(By.xpath("//*[@text='요일별 연재']")).isDisplayed();
}
@AfterClass
public void quit() {
driver.quit();
}
}
Appium](https://image.slidesharecdn.com/26automation-181102022301/85/26-Evolution-of-Android-Automation-Test-16-320.jpg)
![public class TestClass {
public AppiumDriver<WebElement> driver;
@BeforeClass
public void setup() throws Exception {
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability(MobileCapabilityType.PLATFORM_NAME, "Android");
capabilities.setCapability(MobileCapabilityType.UDID, "UDID");
capabilities.setCapability(MobileCapabilityType.PLATFORM_VERSION,"platform-version");
capabilities.setCapability(MobileCapabilityType.AUTOMATION_NAME, "UIAutomator2");
capabilities.setCapability(AndroidMobileCapabilityType.APP_PACKAGE, "package");
capabilities.setCapability(AndroidMobileCapabilityType.APP_ACTIVITY, "launchable-activity");
driver = new AppiumDriver<WebElement>(new URL("http://127.0.0.1:4723/wd/hub"), capabilities);
driver.manage().timeouts().implicitlyWait(15, TimeUnit.SECONDS);
}
@Test
public void scenario_01() {
driver.findElement(By.id("tv_home_weekly_schedule_title")).click();
}
@Test
public void scenario_02() {
driver.findElement(By.xpath("//*[@text='요일별 연재']")).isDisplayed();
}
@AfterClass
public void quit() {
driver.quit();
}
}
Appium](https://image.slidesharecdn.com/26automation-181102022301/85/26-Evolution-of-Android-Automation-Test-17-320.jpg)
![public class TestClass {
public AppiumDriver<WebElement> driver;
@BeforeClass
public void setup() throws Exception {
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability(MobileCapabilityType.PLATFORM_NAME, "Android");
capabilities.setCapability(MobileCapabilityType.UDID, "UDID");
capabilities.setCapability(MobileCapabilityType.PLATFORM_VERSION,"platform-version");
capabilities.setCapability(MobileCapabilityType.AUTOMATION_NAME, "UIAutomator2");
capabilities.setCapability(AndroidMobileCapabilityType.APP_PACKAGE, "package");
capabilities.setCapability(AndroidMobileCapabilityType.APP_ACTIVITY, "launchable-activity");
driver = new AppiumDriver<WebElement>(new URL("http://127.0.0.1:4723/wd/hub"), capabilities);
driver.manage().timeouts().implicitlyWait(15, TimeUnit.SECONDS);
}
@Test
public void scenario_01() {
driver.findElement(By.id("tv_home_weekly_schedule_title")).click();
}
@Test
public void scenario_02() {
driver.findElement(By.xpath("//*[@text='요일별 연재']")).isDisplayed();
}
@AfterClass
public void quit() {
driver.quit();
}
}
Appium](https://image.slidesharecdn.com/26automation-181102022301/85/26-Evolution-of-Android-Automation-Test-18-320.jpg)
![public class TestClass {
public AppiumDriver<WebElement> driver;
@BeforeClass
public void setup() throws Exception {
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability(MobileCapabilityType.PLATFORM_NAME, "Android");
capabilities.setCapability(MobileCapabilityType.UDID, "UDID");
capabilities.setCapability(MobileCapabilityType.PLATFORM_VERSION,"platform-version");
capabilities.setCapability(MobileCapabilityType.AUTOMATION_NAME, "UIAutomator2");
capabilities.setCapability(AndroidMobileCapabilityType.APP_PACKAGE, "package");
capabilities.setCapability(AndroidMobileCapabilityType.APP_ACTIVITY, "launchable-activity");
driver = new AppiumDriver<WebElement>(new URL("http://127.0.0.1:4723/wd/hub"), capabilities);
driver.manage().timeouts().implicitlyWait(15, TimeUnit.SECONDS);
}
@Test
public void scenario_01() {
driver.findElement(By.id("tv_home_weekly_schedule_title")).click();
}
@Test
public void scenario_02() {
driver.findElement(By.xpath("//*[@text='요일별 연재']")).isDisplayed();
}
@AfterClass
public void quit() {
driver.quit();
}
}
Appium](https://image.slidesharecdn.com/26automation-181102022301/85/26-Evolution-of-Android-Automation-Test-19-320.jpg)
![public class TestClass extends Formatter {
public AndroidUtil android;
@BeforeClass
public void setup() {
android = (AndroidUtil) new Automation()
.android()
.mobileApp()
.apkFile("apkPath")
.packageName("Package_Name")
.activityName("Activity_Name")
.logLevel("loglevel")
.start();
}
@Test
public void scenario_01() {
android.click(By.id("tv_home_weekly_schedule_title"));
}
@Test
public void scenario_02() {
android.isElementPresent(By.xpath("//*[@text='요일별 연재']"));
}
@AfterClass
public void quit() {
android.quit();
}
}
Automation Library](https://image.slidesharecdn.com/26automation-181102022301/85/26-Evolution-of-Android-Automation-Test-22-320.jpg)
![public class TestClass extends Formatter {
public AndroidUtil android;
@BeforeClass
public void setup() {
android = (AndroidUtil) new Automation()
.android()
.mobileApp()
.apkFile("apkPath")
.packageName("Package_Name")
.activityName("Activity_Name")
.logLevel("loglevel")
.start();
}
@Test
public void scenario_01() {
android.click(By.id("tv_home_weekly_schedule_title"));
}
@Test
public void scenario_02() {
android.isElementPresent(By.xpath("//*[@text='요일별 연재']"));
}
@AfterClass
public void quit() {
android.quit();
}
}
Automation Library](https://image.slidesharecdn.com/26automation-181102022301/85/26-Evolution-of-Android-Automation-Test-23-320.jpg)
![public class TestClass extends Formatter {
public AndroidUtil android;
@BeforeClass
public void setup() {
android = (AndroidUtil) new Automation()
.android()
.mobileApp()
.apkFile("apkPath")
.packageName("Package_Name")
.activityName("Activity_Name")
.logLevel("loglevel")
.start();
}
@Test
public void scenario_01() {
android.click(By.id("tv_home_weekly_schedule_title"));
}
@Test
public void scenario_02() {
android.isElementPresent(By.xpath("//*[@text='요일별 연재']"));
}
@AfterClass
public void quit() {
android.quit();
}
}
Automation Library](https://image.slidesharecdn.com/26automation-181102022301/85/26-Evolution-of-Android-Automation-Test-24-320.jpg)
![[26]자동화, 계륵에 살 붙이기 : Evolution of Android Automation Test](https://image.slidesharecdn.com/26automation-181102022301/85/26-Evolution-of-Android-Automation-Test-51-320.jpg)
[26]자동화, 계륵에 살 붙이기 : Evolution of Android Automation Test
- 2. Session 6 l 오디오 클립QA 송의초경
- 3. Evolution of Android Automation Test Session 6. 자동화, 계륵에 살 붙이기 N Tech Service SQA l 송의초경
- 4. 발표자 소개 • N Tech Service / SQA • Test Engineer • 네이버 뮤직, 오디오 클립 QA • Automation Library 유지보수
- 5. 제목: 자동화, 계륵에 살 붙이기 1. UI Automation Test 2. Automation Library 3. Android Event Checker 4. Resource Monitoring
- 7. 사용자의 테스트 동작 중 화면상의 마우스 동작, 키보드 입력 등을 자동화 수행이 가능한 script 형태로 변환하여 이후 동일한 형태로 Replay 함으로써 Regression Test를 지원하는 테스트 자동화 방식 UI Automation Test란?
- 8. • 단순하고 반복적인 테스트 • 유지보수 기간 ↑, Test Case ↑ • Side-effect 확인을 위한 회귀 테스트
- 9. Appium 활용한 자동화 사례 Sikuri를 이용한 자동화 사례 Calabash 적용 자동화 사례 ADB를 활용한 자동화 툴 제작 Webkuli 적용사례 Calabash 활용한 자동화 사례 BDD방식 적용 자동화 사례 Uiautomator Stub 활용한 멀티 디바이스 제어 자동화 사례
- 10. 자동화는 문제들을 해결해 주었을까?
- 11. FAIL
- 12. 자동화 구현 및 유지보수 공수 ↑ 정보 공유 부족 Tool 이해도 부족 기대에 미치지 못 하는 Test Coverage
- 14. Robotium UI Automator Espresso Selendroid Calabash Appium Android Yes Yes Yes Yes Yes Yes iOS No No No No Yes Yes Mobile Web Yes (Android) Limited to x.y clicks No Yes Yes (Android) Yes (Android & iOS) Hybrid Limited Limited No Yes No Yes Language Scripting Java Java Java Java, Ruby Ruby Almost Any Supported API levels All 16 => 8, 10, 15-19 10 <= All All Jail Breaking Routing No No No No No No Community Contributors Google Google Active Pretty quiet Active
- 15. Test Script UI Automator DeviceAppium Server JSON wire protocol Appium
- 16. public class TestClass { public AppiumDriver<WebElement> driver; @BeforeClass public void setup() throws Exception { DesiredCapabilities capabilities = new DesiredCapabilities(); capabilities.setCapability(MobileCapabilityType.PLATFORM_NAME, "Android"); capabilities.setCapability(MobileCapabilityType.UDID, "UDID"); capabilities.setCapability(MobileCapabilityType.PLATFORM_VERSION,"platform-version"); capabilities.setCapability(MobileCapabilityType.AUTOMATION_NAME, "UIAutomator2"); capabilities.setCapability(AndroidMobileCapabilityType.APP_PACKAGE, "package"); capabilities.setCapability(AndroidMobileCapabilityType.APP_ACTIVITY, "launchable-activity"); driver = new AppiumDriver<WebElement>(new URL("http://127.0.0.1:4723/wd/hub"), capabilities); driver.manage().timeouts().implicitlyWait(15, TimeUnit.SECONDS); } @Test public void scenario_01() { driver.findElement(By.id("tv_home_weekly_schedule_title")).click(); } @Test public void scenario_02() { driver.findElement(By.xpath("//*[@text='요일별 연재']")).isDisplayed(); } @AfterClass public void quit() { driver.quit(); } } Appium
- 17. public class TestClass { public AppiumDriver<WebElement> driver; @BeforeClass public void setup() throws Exception { DesiredCapabilities capabilities = new DesiredCapabilities(); capabilities.setCapability(MobileCapabilityType.PLATFORM_NAME, "Android"); capabilities.setCapability(MobileCapabilityType.UDID, "UDID"); capabilities.setCapability(MobileCapabilityType.PLATFORM_VERSION,"platform-version"); capabilities.setCapability(MobileCapabilityType.AUTOMATION_NAME, "UIAutomator2"); capabilities.setCapability(AndroidMobileCapabilityType.APP_PACKAGE, "package"); capabilities.setCapability(AndroidMobileCapabilityType.APP_ACTIVITY, "launchable-activity"); driver = new AppiumDriver<WebElement>(new URL("http://127.0.0.1:4723/wd/hub"), capabilities); driver.manage().timeouts().implicitlyWait(15, TimeUnit.SECONDS); } @Test public void scenario_01() { driver.findElement(By.id("tv_home_weekly_schedule_title")).click(); } @Test public void scenario_02() { driver.findElement(By.xpath("//*[@text='요일별 연재']")).isDisplayed(); } @AfterClass public void quit() { driver.quit(); } } Appium
- 18. public class TestClass { public AppiumDriver<WebElement> driver; @BeforeClass public void setup() throws Exception { DesiredCapabilities capabilities = new DesiredCapabilities(); capabilities.setCapability(MobileCapabilityType.PLATFORM_NAME, "Android"); capabilities.setCapability(MobileCapabilityType.UDID, "UDID"); capabilities.setCapability(MobileCapabilityType.PLATFORM_VERSION,"platform-version"); capabilities.setCapability(MobileCapabilityType.AUTOMATION_NAME, "UIAutomator2"); capabilities.setCapability(AndroidMobileCapabilityType.APP_PACKAGE, "package"); capabilities.setCapability(AndroidMobileCapabilityType.APP_ACTIVITY, "launchable-activity"); driver = new AppiumDriver<WebElement>(new URL("http://127.0.0.1:4723/wd/hub"), capabilities); driver.manage().timeouts().implicitlyWait(15, TimeUnit.SECONDS); } @Test public void scenario_01() { driver.findElement(By.id("tv_home_weekly_schedule_title")).click(); } @Test public void scenario_02() { driver.findElement(By.xpath("//*[@text='요일별 연재']")).isDisplayed(); } @AfterClass public void quit() { driver.quit(); } } Appium
- 19. public class TestClass { public AppiumDriver<WebElement> driver; @BeforeClass public void setup() throws Exception { DesiredCapabilities capabilities = new DesiredCapabilities(); capabilities.setCapability(MobileCapabilityType.PLATFORM_NAME, "Android"); capabilities.setCapability(MobileCapabilityType.UDID, "UDID"); capabilities.setCapability(MobileCapabilityType.PLATFORM_VERSION,"platform-version"); capabilities.setCapability(MobileCapabilityType.AUTOMATION_NAME, "UIAutomator2"); capabilities.setCapability(AndroidMobileCapabilityType.APP_PACKAGE, "package"); capabilities.setCapability(AndroidMobileCapabilityType.APP_ACTIVITY, "launchable-activity"); driver = new AppiumDriver<WebElement>(new URL("http://127.0.0.1:4723/wd/hub"), capabilities); driver.manage().timeouts().implicitlyWait(15, TimeUnit.SECONDS); } @Test public void scenario_01() { driver.findElement(By.id("tv_home_weekly_schedule_title")).click(); } @Test public void scenario_02() { driver.findElement(By.xpath("//*[@text='요일별 연재']")).isDisplayed(); } @AfterClass public void quit() { driver.quit(); } } Appium
- 20. Automation Library • 자동화 테스트 환경 Setting • 자동화 구현 패턴 표준화 • Appium Client API의 사용성 • 객관적 지표 산출 고려 사항
- 22. public class TestClass extends Formatter { public AndroidUtil android; @BeforeClass public void setup() { android = (AndroidUtil) new Automation() .android() .mobileApp() .apkFile("apkPath") .packageName("Package_Name") .activityName("Activity_Name") .logLevel("loglevel") .start(); } @Test public void scenario_01() { android.click(By.id("tv_home_weekly_schedule_title")); } @Test public void scenario_02() { android.isElementPresent(By.xpath("//*[@text='요일별 연재']")); } @AfterClass public void quit() { android.quit(); } } Automation Library
- 23. public class TestClass extends Formatter { public AndroidUtil android; @BeforeClass public void setup() { android = (AndroidUtil) new Automation() .android() .mobileApp() .apkFile("apkPath") .packageName("Package_Name") .activityName("Activity_Name") .logLevel("loglevel") .start(); } @Test public void scenario_01() { android.click(By.id("tv_home_weekly_schedule_title")); } @Test public void scenario_02() { android.isElementPresent(By.xpath("//*[@text='요일별 연재']")); } @AfterClass public void quit() { android.quit(); } } Automation Library
- 24. public class TestClass extends Formatter { public AndroidUtil android; @BeforeClass public void setup() { android = (AndroidUtil) new Automation() .android() .mobileApp() .apkFile("apkPath") .packageName("Package_Name") .activityName("Activity_Name") .logLevel("loglevel") .start(); } @Test public void scenario_01() { android.click(By.id("tv_home_weekly_schedule_title")); } @Test public void scenario_02() { android.isElementPresent(By.xpath("//*[@text='요일별 연재']")); } @AfterClass public void quit() { android.quit(); } } Automation Library
- 27. • 자동화 시나리오 진행 중 안정성 문제 • UI Assertion 대한 신뢰성
- 28. • 재생 중 일까? • Audio Focus는 가져왔을까?
- 29. • 자동화 테스트 중 Interrupt • 알 수 없는 간헐적 Fail Result
- 31. • Log 출력 방식 • Monitoring Start/ Stop • Event Log Event Checker
- 32. Service Command Broadcast Receiver Event Checker
- 33. Sensor • 기울기 센서를 이용하여 orientation 전환 확인 Audio Focus, Audio Manager • Audio Focus 선점 및 Audio 재생 확인 Notification listener • Application Notification 정보 수집 Event Checker
- 34. • 약 30개 Event • 약 80개 Log Event Call Doze LTE SMS USB/AC charging WiFi GPS ScreenON/OFF AirPlane Bluetooth is Plug Roaming Battery Save Lock Screen BT Connection Volume Ringer Mode Locale Change NFC BT Discover Mode Do Not Distrurb Mode Picture Data Saver Mode BT Discover Noisy HeadSet Plug Boot KeyBoardChange Screen Orientation Audio PlayBack Dock Event Checker
- 35. Test Script Android EventChecker Event Checker
- 36. Event Checker
- 37. Event Checker
- 39. 동일 시간동일 스텝 반복적 Resource Monitoring
- 40. • Battery 소모량 LCD/OLED & CPU & WiFi & 3G GPS & Audio CPU 사용률 (%) Memory 사용량 PSS & Private Dirty & Shared Dirty 발생 Traffic TX & RX (Upload & Download) * 모든 항목을 개별, 혹은 전체로 측정 가능 rMon Mobile Version Resource Monitoring
- 41. Android 5.1.1 이후 사용 불가 Resource Monitoring • 신규 App Launching & 개편 • 유사 서비스와 BMT
- 42. Resource Monitoring rMon PC Version adb shell /data/local/rmon • ADB를 활용한 Resource 측정 방식 • Android Application의 소모 CPU, Memory, Traffic 측정 • Android 단말기 내부에 rMon 명령어 추가 • ADB > rMon 명령어 호출하여 리소스 데이터 수집
- 45. 리소스 측정 History 관리 Resource Monitoring
- 47. Android Automation Test Maven Repository Code RepositoryCoding CI 서버 Target DeviceResult Report • GitLab으로 Automation Test Project 관리 • CI서버 통해 Daily Build 수행
- 48. PC Browser & iOS nMobile
- 49. 모바일 기기 원격 제어 솔루션 • 화면 제어, 디버깅 등 • Real Time에 가까운 응답속도 • 실제 단말과 동일한 사용감을 제공 nMobile 자동화 연동 시 장점 • 가상 단말 아닌 리얼 디바이스 대상 수행 가능 • 다양한 단말 보유 (버전 별, 제조사 별, 국가 별) • ADB 연결 가능 / 다양한 API 지원 • 물리적인 연결 없이 접근 가능 / 사용 용이성 nMobile
- 50. PC Browser & iOS nMobile
- 52. Android Automation Test • Library를 이용한 안정적인 자동화 프로젝트 생성 • 외부 인터럽트 및 기능 Assertion에 대한 단점 보완 • 자동화 특성 반영 Resource Monitoring Result
- 53. Thank You.