SlideShare a Scribd company logo

DEVIEW2013: Automating Performance Tests for Android Applications

Download as PPTX, PDF
Kyungmin Lee, profile picture
Kyungmin Lee

The document discusses automating performance testing of Android apps. It explains that instrumentation can be used to track activity navigation and lifecycle without modifying the app or framework. The kmemtracer tool uses instrumentation to trace memory usage historically. It provides steps to use kmemtracer, including installing the app and test packages, starting instrumentation, interacting with the app, and pulling the trace file. The document also discusses what aspects of performance to measure, like memory, CPU and network, and some public and internal Android/Linux commands that can be used to measure resources.

Technology
Related topics:
Software Testing Insights • Performance Testing
1 of 169
Downloaded 173 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
안드로이드 앱의 성능 테스트, 자동화가 가능할까? 이경민 / LG전자
About Me
About Me Android Platform
About Me Android Platform SDK
About Me Android Platform SDET SDK
About Me webOS Android Platform SDET SDK
About Me webOS Android Platform SDET Test Automation SDK
About Me webOS kandroid.org Android Platform SDET Test Automation SDK
About Me webOS kandroid.org Android Platform SDET SDK Test Automation snailee
CONTENTS Why Automated Performance Testing? How to Automate Performance Testing? - Historical Analysis w/ kmemtracer - Measuring Performance - Controlling UI
Why Automated Performance Testing?
If you can’t measure it, you can’t manage it. Peter F. Drucker (1909~2005)
If you can’t measure it, you can’t manage it. Peter F. Drucker (1909~2005) If you can’t automate the measurement of it, it is difficult to improve it.
Diversity or Fragmentation Platform Versions http://developer.android.com/about/dashboards/index.html
Diversity or Fragmentation Platform Versions Screen Sizes and Densities http://developer.android.com/about/dashboards/index.html
http://www.mactrast.com/2012/05/visualized-the-truth-about-android-fragmentation/
How to Automate Performance Testing? - Historical Analysis w/ kmemtracer - Measuring Performance - Controlling UI
Why Historical Analysis? Source: Memory and Performance at the 11th Korea Android Conference
Why Historical Analysis? Source: Memory and Performance at the 11th Korea Android Conference
Why Historical Analysis? Activity Navigation & Lifecycle Source: Memory and Performance at the 11th Korea Android Conference
How to Track Activity Navigation & Lifecycle?
How to Track Activity Navigation & Lifecycle? Override onCreate, onStart, … of your Activity classes
How to Track Activity Navigation & Lifecycle? Modify android.app.Activity in the framework
How to Track Activity Navigation & Lifecycle? Is there a way that does not require the modification of an application or framework?
How to Track Activity Navigation & Lifecycle? Is there a way that does not require the modification of an application or framework? Instrumentation
DEVIEW2013: Automating Performance Tests for Android Applications
Android instrumentation is a set of control methods or "hooks" in the Android system. These hooks control an Android component independently of its normal lifecycle. They also control how Android loads applications. (http://developer.android.com/tools/testing/testing_android.html)
Historical Analysis w/ kmemtracer Application Package Control android.test. Instrumentation TestRunner Test Package
Historical Analysis w/ kmemtracer https://github.com/snailee/kmemtracer-libs Application Package Control Track android.test. Instrumentation TestRunner org.kandroid. memtracer.Memory Instrumentation Test Package Trace Package
android.app.Instrumentation void void void void void void void void void void void void void callActivityOnCreate(Activity, Bundle) callActivityOnDestroy(Activity) callActivityOnNewIntent(Activity, Intent) callActivityOnPause(Activity) callActivityOnPostCreate(Activity, Bundle) callActivityOnRestart(Activity) callActivityOnRestoreInstanceState(Activity, Bundle) callActivityOnResume(Activity) callActivityOnSaveInstanceState(Activity, Bundle) callActivityOnStart(Activity) callActivityOnStop(Activity) callActivityOnUserLeaving(Activity) callApplicationOnCreate(Application)
android.app.Instrumentation void void void void void void void void void void void void void callActivityOnCreate(Activity, Bundle) callActivityOnDestroy(Activity) callActivityOnNewIntent(Activity, Intent) callActivityOnPause(Activity) callActivityOnPostCreate(Activity, Bundle) callActivityOnRestart(Activity) callActivityOnRestoreInstanceState(Activity, Bundle) callActivityOnResume(Activity) callActivityOnSaveInstanceState(Activity, Bundle) callActivityOnStart(Activity) callActivityOnStop(Activity) callActivityOnUserLeaving(Activity) callApplicationOnCreate(Application) Call for Control
android.app.Instrumentation void void void void void void void void void void void void void callActivityOnCreate(Activity, Bundle) callActivityOnDestroy(Activity) callActivityOnNewIntent(Activity, Intent) callActivityOnPause(Activity) callActivityOnPostCreate(Activity, Bundle) callActivityOnRestart(Activity) callActivityOnRestoreInstanceState(Activity, Bundle) callActivityOnResume(Activity) callActivityOnSaveInstanceState(Activity, Bundle) callActivityOnStart(Activity) callActivityOnStop(Activity) callActivityOnUserLeaving(Activity) callApplicationOnCreate(Application) Call for Control Override for Track
org.kandroid.memtracer.MemoryInstrumentation public class MemoryInstrumentation extends Instrumentation { private static final String ARGUMENT_INSTRUMENT_CLASS = "class"; private MemoryTracer mMemoryTracer; private String mMainActivityName; ... @Override public void onCreate(Bundle arguments) { mMainActivityName = arguments.getString(ARGUMENT_INSTRUMENT_CLASS); mMemoryTracer = createMemoryTracer(); mMemoryTracer.startTracing(getTargetContext().getPackageName()); start(); } @Override public void callActivityOnCreate(Activity activity, Bundle icicle) { String tag = activity.getLocalClassName()+"-OnCreate"; Bundle snapshot = mMemoryTracer.addSnapshot(tag); super.callActivityOnCreate(activity, icicle); }
How to Use kmemtracer?
How to Use kmemtracer? 1. Create an Android Test Project for the trace package.
How to Use kmemtracer? 1. Create an Android Test Project for the trace package. 2. Add ‘kmemtracer-libs.jar’ in the ‘libs’ directory.
How to Use kmemtracer? 1. Create an Android Test Project for the trace package. 2. Add ‘kmemtracer-libs.jar’ in the ‘libs’ directory. 3. Edit the <instrumentation> element in the Manifest file. <manifest xmlns:android="http://schemas.android.com/apk/res/android" package="com.example.android.apis.test" android:versionCode="1" android:versionName="1.0"> <instrumentation android:name="org.kandroid.memtracer.MemoryInstrumentation" android:targetPackage="com.example.android.apis" /> <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"/> <application android:label="@string/app_name" android:icon="@drawable/ic_launcher“> <uses-library android:theme="android.text.runner“> </application> </manifest>
How to Use kmemtracer? 4. Install the trace package as well as the application package. $ adb install –r ApiDemos.apk $ adb install –r ApiDemosTest.apk
How to Use kmemtracer? 4. Install the trace package as well as the application package. $ adb install –r ApiDemos.apk $ adb install –r ApiDemosTest.apk 5. Start the instrumentation with ‘am instrument’ in the shell. $ > > > adb shell am instrument -e class com.example.android.apis.ApiDemos com.example.android.apis.tests/ org.kandroid.memtracer.MemoryInstrumentation
How to Use kmemtracer? 4. Install the trace package as well as the application package. $ adb install –r ApiDemos.apk $ adb install –r ApiDemosTest.apk 5. Start the instrumentation with ‘am instrument’ in the shell. $ > > > adb shell am instrument -e class com.example.android.apis.ApiDemos com.example.android.apis.tests/ org.kandroid.memtracer.MemoryInstrumentation 6. Do interact with the application.
How to Use kmemtracer? 4. Install the trace package as well as the application package. $ adb install –r ApiDemos.apk $ adb install –r ApiDemosTest.apk 5. Start the instrumentation with ‘am instrument’ in the shell. $ > > > adb shell am instrument -e class com.example.android.apis.ApiDemos com.example.android.apis.tests/ org.kandroid.memtracer.MemoryInstrumentation 6. Do interact with the application. 7. Pull the trace file ‘kmemtrace.csv’ from /sdcard/kmemtracer. $ adb pull /sdcard/kmemtracer/kmemtrace.csv .
How to Use kmemtracer? 8. Open the trace file with Excel and create charts.
How to Automate Performance Testing? - Historical Analysis w/ kmemtracer - Measuring Performance - Controlling UI
What to Measure? Rendering (or GUI) Performance Frame Rate (FPS), Frame Time , Jankiness, Input Latency, … Resource Consumption Memory, CPU, I/O (Disk, Network), Battery, … Network Performance Response time, Throughput, … …
What to Measure? Rendering (or GUI) Performance Frame Rate (FPS), Frame Time , Jankiness, Input Latency, … Resource Consumption Memory, CPU, I/O (Disk, Network), Battery, … Network Performance Response time, Throughput, … …
How to Measure Resource Consumption?
How to Measure Resource Consumption? API cmds
How to Measure Resource Consumption? cmds API Public Internal
How to Measure Resource Consumption? cmds API Public Internal Android Linux
How to Measure Resource Consumption? cmds API Public Easy Internal Android Linux Flexible
Measuring Memory Usage
Measuring Memory Usage Public android.app.ActivityManager.MemoryInfo android.os.Debug android.os.Debug.MemoryInfo
Measuring Memory Usage Public Internal android.app.ActivityManager.MemoryInfo android.os.Debug android.os.Debug.MemoryInfo dalvik.system.VMDebug
Measuring Memory Usage Public Internal Android android.app.ActivityManager.MemoryInfo android.os.Debug android.os.Debug.MemoryInfo dalvik.system.VMDebug adb shell dumpsys meminfo [pid|pname] adb shell procrank
Measuring Memory Usage Public Internal Android Linux android.app.ActivityManager.MemoryInfo android.os.Debug android.os.Debug.MemoryInfo dalvik.system.VMDebug adb shell dumpsys meminfo [pid|pname] adb shell procrank adb shell cat /proc/meminfo adb shell cat /proc/<pid>/[s]maps adb shell cat /proc/vmstat adb shell top adb shell ps
Measuring CPU Usage
Measuring CPU Usage Public android.os.Process android.os.Debug.InstructionCount
Measuring CPU Usage Public Internal android.os.Process android.os.Debug.InstructionCount com.android.internal.os.ProcessStats
Measuring CPU Usage Public Internal Android android.os.Process android.os.Debug.InstructionCount com.android.internal.os.ProcessStats adb shell dumpsys cpuinfo
Measuring CPU Usage Public Internal Android android.os.Process android.os.Debug.InstructionCount com.android.internal.os.ProcessStats adb shell dumpsys cpuinfo adb shell cat /proc/stat adb shell top Linux
Measuring I/O Usage
Measuring I/O Usage Public android.net.TrafficStats
Measuring I/O Usage Public Internal android.net.TrafficStats android.net.NetworkStats android.net.NetworkStatsHistory
Measuring I/O Usage Public Internal Android android.net.TrafficStats android.net.NetworkStats android.net.NetworkStatsHistory adb shell dumpsys netstats adb shell dumpsys diskstats
Measuring I/O Usage Public Internal Android android.net.TrafficStats android.net.NetworkStats android.net.NetworkStatsHistory adb shell dumpsys netstats adb shell dumpsys diskstats /proc/net/* Linux
Measuring Battery Usage
Measuring Battery Usage Public android.os.BatteryManager
Measuring Battery Usage Public Internal android.os.BatteryManager android.os.BatteryStats com.android.internal.os.PowerProfile
Measuring Battery Usage Public Internal Android android.os.BatteryManager android.os.BatteryStats com.android.internal.os.PowerProfile adb shell dumpsys battery adb shell dumpsys batteryinfo
Measuring Battery Usage Public Internal Android android.os.BatteryManager android.os.BatteryStats com.android.internal.os.PowerProfile adb shell dumpsys battery adb shell dumpsys batteryinfo None (?) Linux
Tracing Performance Snapshots in kmemtracer public class MemoryInstrumentation extends Instrumentation {
Tracing Performance Snapshots in kmemtracer public class MemoryInstrumentation extends Instrumentation { ... private MemoryTracer mMemoryTracer; ... @Override public void onCreate(Bundle arguments) { ... mMemoryTracer = createMemoryTracer(); ... } ... Create an instance of MemoryTracer
Tracing Performance Snapshots in kmemtracer public class MemoryInstrumentation extends Instrumentation { ... private MemoryTracer mMemoryTracer; ... @Override public void onCreate(Bundle arguments) { ... mMemoryTracer = createMemoryTracer(); ... } ... @Override public void callActivityOnCreate(Activity activity, Bundle icicle) { String tag = activity.getLocalClassName()+"-OnCreate"; Bundle snapshot = mMemoryTracer.addSnapshot(tag); ... } ... } Create an instance of MemoryTracer Add performance snapshot
MemoryTracer$addSnapshot public Bundle addSnapshot(String label) {
MemoryTracer$addSnapshot public Bundle addSnapshot(String label) { ... long nativeMax = Debug.getNativeHeapSize() / 1024; long nativeAllocated = Debug.getNativeHeapAllocatedSize() / 1024; long nativeFree = Debug.getNativeHeapFreeSize() / 1024; Debug.MemoryInfo memInfo = new Debug.MemoryInfo(); Debug.getMemoryInfo(memInfo); ... Get usage data
MemoryTracer$addSnapshot public Bundle addSnapshot(String label) { ... long nativeMax = Debug.getNativeHeapSize() / 1024; long nativeAllocated = Debug.getNativeHeapAllocatedSize() / 1024; long nativeFree = Debug.getNativeHeapFreeSize() / 1024; Debug.MemoryInfo memInfo = new Debug.MemoryInfo(); Debug.getMemoryInfo(memInfo); ... Bundle snapshot = new Bundle(); snapshot.putString(METRIC_KEY_LABEL, label); ... snapshot.putLong(METRIC_KEY_NATIVE_SIZE, nativeMax); snapshot.putLong(METRIC_KEY_NATIVE_ALLOCATED, nativeAllocated); snapshot.putLong(METRIC_KEY_NATIVE_FREE, nativeFree); ... Get usage data Save data to a bundle
MemoryTracer$addSnapshot public Bundle addSnapshot(String label) { ... long nativeMax = Debug.getNativeHeapSize() / 1024; long nativeAllocated = Debug.getNativeHeapAllocatedSize() / 1024; long nativeFree = Debug.getNativeHeapFreeSize() / 1024; Debug.MemoryInfo memInfo = new Debug.MemoryInfo(); Debug.getMemoryInfo(memInfo); ... Bundle snapshot = new Bundle(); snapshot.putString(METRIC_KEY_LABEL, label); ... snapshot.putLong(METRIC_KEY_NATIVE_SIZE, nativeMax); snapshot.putLong(METRIC_KEY_NATIVE_ALLOCATED, nativeAllocated); snapshot.putLong(METRIC_KEY_NATIVE_FREE, nativeFree); ... if (mResultsWriter != null) { mResultsWriter.writeTraceSnapshot(snapshot); } return snapshot; } Get usage data Save data to a bundle Write date using ResultsWriter
MemoryTracerCsvWriter$writeTraceSnapshot public class MemoryTraceCsvWriter implements MemoryTracer.ResultsWriter {
MemoryTracerCsvWriter$writeTraceSnapshot public class MemoryTraceCsvWriter implements MemoryTracer.ResultsWriter { ... public MemoryTraceCsvWriter(String[] metricKeys) { mMetricKeys = metricKeys; } ... Create an instance with metrics to print
MemoryTracerCsvWriter$writeTraceSnapshot public class MemoryTraceCsvWriter implements MemoryTracer.ResultsWriter { ... public MemoryTraceCsvWriter(String[] metricKeys) { mMetricKeys = metricKeys; } ... private void openTraceFile(String filename) { ... mTraceOut = new PrintStream(new FileOutputStream(traceFile)); ... } ... Create an instance with metrics to print Open a trace file in the sdcard at init
MemoryTracerCsvWriter$writeTraceSnapshot public class MemoryTraceCsvWriter implements MemoryTracer.ResultsWriter { ... public MemoryTraceCsvWriter(String[] metricKeys) { mMetricKeys = metricKeys; } ... private void openTraceFile(String filename) { ... mTraceOut = new PrintStream(new FileOutputStream(traceFile)); ... } ... @Override public void writeTraceSnapshot(Bundle snapshot) { PrintStream out = mTraceOut; for (String key : mMetricKeys) { out.print(snapshot.get(key)); out.print(','); } out.println(); out.flush(); } ... } Create an instance with metrics to print Open a trace file in the sdcard at init Write snapshot to a line of the file
To Do’s
To Do’s Add more tracers CpuTracer IOTracer BatteryTracer
To Do’s Add more tracers CpuTracer IOTracer BatteryTracer Make metrics to gather configurable
To Do’s Add more tracers CpuTracer IOTracer BatteryTracer Make metrics to gather configurable Make trace output destination configurable Trace out to internal file (not on sdcard) Trace out to an analytics server
To Do’s Add more tracers CpuTracer IOTracer BatteryTracer Make metrics to gather configurable Make trace output destination configurable Trace out to internal file (not on sdcard) Trace out to an analytics server Welcome contributions https://github.com/snailee/kmemtracer-libs
How to Automate Performance Testing? - Historical Analysis w/ kmemtracer - Measuring Performance - Controlling UI
Automate Controlling UI Application Package Control Track android.test. Instrumentation TestRunner org.kandroid. memtracer.Memory Instrumentation Test Package Trace Package
Automate Controlling UI Application Package Control Track android.test. Instrumentation TestRunner org.kandroid. memtracer.Memory Instrumentation Test Package Trace Package
Automate Controlling UI Application Package Control Track android.test. Instrumentation TestRunner org.kandroid. memtracer.Memory Instrumentation Test Package Trace Package
Three Alternatives
Three Alternatives Instrumentation API Level 1 ~ Single activity Framework APIs White-box http://developer.android.com/tools/testing/testing_android.html
Three Alternatives Instrumentation API Level 1 ~ Single activity Framework APIs White-box Robotium 2010/01 ~ Multiple activities High-level APIs White-box & black-box http://developer.android.com/tools/testing/testing_android.html https://code.google.com/p/robotium/
Three Alternatives Instrumentation API Level 1 ~ Single activity Framework APIs White-box Robotium uiautomator 2010/01 ~ Multiple activities API Level 17 ~ High-level APIs White-box & black-box Multiple processes Simple APIs Black-box http://developer.android.com/tools/testing/testing_android.html https://code.google.com/p/robotium/ http://developer.android.com/tools/testing/testing_ui.html
Case Study: ApiDemos https://github.com/snailee/deview-2013-samples
Case Study: ApiDemos https://github.com/snailee/deview-2013-samples Click
Case Study: ApiDemos https://github.com/snailee/deview-2013-samples Click
Case Study: ApiDemos https://github.com/snailee/deview-2013-samples Click Scroll down
Case Study: ApiDemos https://github.com/snailee/deview-2013-samples Click Scroll down
Case Study: ApiDemos https://github.com/snailee/deview-2013-samples Click Scroll down Click
Case Study: ApiDemos https://github.com/snailee/deview-2013-samples Click Scroll down Click
Case Study: ApiDemos https://github.com/snailee/deview-2013-samples Click Scroll down Click Click
Case Study: ApiDemos https://github.com/snailee/deview-2013-samples Click Scroll down Click Click
Controlling UI Using Instrumentation
Controlling UI Using Instrumentation 1. Create an Android Test Project for the test package. $ android create test-project --main ../ApiDemos > –-name ApiDemosInstrumentTest --path ApiDemosInstrumentTest
Controlling UI Using Instrumentation 1. Create an Android Test Project for the test package. $ android create test-project --main ../ApiDemos > –-name ApiDemosInstrumentTest --path ApiDemosInstrumentTest 2. Add a test class inheriting ActivityInstrumentationTestCase2. public class ApiDemosInstrumentTest extends ActivityInstrumentationTestCase2<ApiDemos> { ... public ApiDemosInstrumentTest() { super(ApiDemos.class); }
Controlling UI Using Instrumentation 1. Create an Android Test Project for the test package. $ android create test-project --main ../ApiDemos > –-name ApiDemosInstrumentTest --path ApiDemosInstrumentTest 2. Add a test class inheriting ActivityInstrumentationTestCase2. public class ApiDemosInstrumentTest extends ActivityInstrumentationTestCase2<ApiDemos> { ... public ApiDemosInstrumentTest() { super(ApiDemos.class); } 3. Add a test method and launch the activity. public void testNavigate() { final ApiDemos apiDemos = getActivity(); ... }
Controlling UI Using Instrumentation 4. Find and act on a control. public void testNavigate() {
Controlling UI Using Instrumentation 4. Find and act on a control. Find a view to control public void testNavigate() { ... final ListView demoList = apiDemos.getListView(); final View graphicsDemo = getChildViewByText(demoList, "Graphics"); assertNotNull("Graphics demo should exist", graphicsDemo);
Controlling UI Using Instrumentation 4. Find and act on a control. Find a view to control public void testNavigate() { ... final ListView demoList = apiDemos.getListView(); final View graphicsDemo = getChildViewByText(demoList, "Graphics"); assertNotNull("Graphics demo should exist", graphicsDemo); apiDemos.runOnUiThread(new Runnable() { public void run() { int pos = demoList.getPositionForView(graphicsDemo); demoList.smoothScrollToPosition(pos); demoList.setSelection(pos); demoList.performItemClick(graphicsDemo, pos, graphicsDemo.getId()); } }); ... Act on the view
Controlling UI Using Instrumentation 4. Find and act on a control. Find a view to control public void testNavigate() { ... final ListView demoList = apiDemos.getListView(); final View graphicsDemo = getChildViewByText(demoList, "Graphics"); assertNotNull("Graphics demo should exist", graphicsDemo); apiDemos.runOnUiThread(new Runnable() { public void run() { int pos = demoList.getPositionForView(graphicsDemo); demoList.smoothScrollToPosition(pos); demoList.setSelection(pos); demoList.performItemClick(graphicsDemo, pos, graphicsDemo.getId()); } }); ... Act on the view
Controlling UI Using Instrumentation 4. Find and act on a control: getChildViewByText. private View getChildViewByText(ListView listView, String text) {
Controlling UI Using Instrumentation 4. Find and act on a control: getChildViewByText. private View getChildViewByText(ListView listView, String text) { View view = null; int count = listView.getCount(); Log.e(TAG, “no. children = " + count); for (int i = 0; i < count; i++) { TextView textView = (TextView) listView.getChildAt(i); Log.e(TAG, i + "-th text view is " + textView); Iterate through child views
Controlling UI Using Instrumentation 4. Find and act on a control: getChildViewByText. private View getChildViewByText(ListView listView, String text) { View view = null; int count = listView.getCount(); Log.e(TAG, “no. children = " + count); for (int i = 0; i < count; i++) { TextView textView = (TextView) listView.getChildAt(i); Log.e(TAG, i + "-th text view is " + textView); if (textView != null && textView.getText().equals(text)) { view = textView; break; } Iterate through child views Compare texts
Controlling UI Using Instrumentation 4. Find and act on a control: getChildViewByText. private View getChildViewByText(ListView listView, String text) { View view = null; int count = listView.getCount(); Log.e(TAG, “no. children = " + count); for (int i = 0; i < count; i++) { TextView textView = (TextView) listView.getChildAt(i); Log.e(TAG, i + "-th text view is " + textView); if (textView != null && textView.getText().equals(text)) { view = textView; break; } } return view; } Iterate through child views Compare texts
Controlling UI Using Instrumentation 4. Find and act on a control: getChildViewByText. private View getChildViewByText(ListView listView, String text) { View view = null; int count = listView.getCount(); Log.e(TAG, “no. children = " + count); for (int i = 0; i < count; i++) { TextView textView = (TextView) listView.getChildAt(i); Log.e(TAG, i + "-th text view is " + textView); if (textView != null && textView.getText().equals(text)) { view = textView; break; } } return view; } com.example.android.apis.instrument.test.ApiDemosInstrumentTest: INSTRUMENTATION_RESULT: shortMsg=junit.framework.AssertionFailedError INSTRUMENTATION_RESULT: longMsg= junit.framework.AssertionFailedError: OpenGL ES demo should exist INSTRUMENTATION_CODE: 0 Iterate through child views Compare texts
Controlling UI Using Instrumentation 4. Find and act on a control: Revised getChildViewByText. private View getChildViewByText(final ListView listView, String text) { ... for (int i = 0; i < count; i++) { TextView textView = (TextView) listView.getChildAt(i); if (textView != null && textView.getText().equals(text)) { view = textView; break; } if (textView == null) { final int lastPos = listView.getLastVisiblePosition(); getInstrumentation().runOnMainSync(new Runnable() { public void run() { listView.smoothScrollToPosition(lastPos + 5, lastPos); } }); getInstrumentation().waitForIdleSync(); i = listView.getFirstVisiblePosition(); } } return view; }
Controlling UI Using Instrumentation 4. Find and act on a control: Revised getChildViewByText. private View getChildViewByText(final ListView listView, String text) { ... for (int i = 0; i < count; i++) { TextView textView = (TextView) listView.getChildAt(i); if (textView != null && textView.getText().equals(text)) { view = textView; break; } if (textView == null) { final int lastPos = listView.getLastVisiblePosition(); getInstrumentation().runOnMainSync(new Runnable() { public void run() { listView.smoothScrollToPosition(lastPos + 5, lastPos); } }); getInstrumentation().waitForIdleSync(); i = listView.getFirstVisiblePosition(); } } return view; } Scroll down
Controlling UI Using Instrumentation 5. Monitor a new Activity to launch. public void testNavigate() { ... Instrumentation.ActivityMonitor graphicsDemoActivityMonitor = getInstrumentation().addMonitor( ApiDemos.class.getName(), null, false); final ApiDemos graphicsDemoActivity = (ApiDemos) graphicsDemoActivityMonitor.waitForActivity(); assertNotNull("Graphics Demo (ApiDemos) activity should not be null", graphicsDemoActivity); ...
Controlling UI Using Instrumentation 5. Monitor a new Activity to launch. public void testNavigate() { ... Instrumentation.ActivityMonitor graphicsDemoActivityMonitor = getInstrumentation().addMonitor( ApiDemos.class.getName(), null, false); final ApiDemos graphicsDemoActivity = (ApiDemos) graphicsDemoActivityMonitor.waitForActivity(); assertNotNull("Graphics Demo (ApiDemos) activity should not be null", graphicsDemoActivity); ... 6. Iterate step 4 & 5.
Controlling UI Using Instrumentation 7. Build and Install the test package. $ cd ApiDemosInstrumentTest $ ant debug $ adb install –r bin/ApiDemosInstrumentTest-debug.apk
Controlling UI Using Instrumentation 7. Build and Install the test package. $ cd ApiDemosInstrumentTest $ ant debug $ adb install –r bin/ApiDemosInstrumentTest-debug.apk 8. Run the test and see the result. $ adb shell am instrument -w > com.example.android.apis.instrument.test/ > android.test.InstrumentationTestRunner
Controlling UI Using Instrumentation 7. Build and Install the test package. $ cd ApiDemosInstrumentTest $ ant debug $ adb install –r bin/ApiDemosInstrumentTest-debug.apk 8. Run the test and see the result. $ adb shell am instrument -w > com.example.android.apis.instrument.test/ > android.test.InstrumentationTestRunner $ ant test
Controlling UI Using Robotium
Controlling UI Using Robotium 1. Create an Android Test Project for the test package. $ android create test-project --main ../ApiDemos > –-name ApiDemosRobotiumTest --path ApiDemosRobotiumTest
Controlling UI Using Robotium 1. Create an Android Test Project for the test package. $ android create test-project --main ../ApiDemos > –-name ApiDemosRobotiumTest --path ApiDemosRobotiumTest 2. Add ‘robotium-solo-<ver>.jar’ in the ‘libs’ directory.
Controlling UI Using Robotium 1. Create an Android Test Project for the test package. $ android create test-project --main ../ApiDemos > –-name ApiDemosRobotiumTest --path ApiDemosRobotiumTest 2. Add ‘robotium-solo-<ver>.jar’ in the ‘libs’ directory. 3. Add a test class inheriting ActivityInstrumentationTestCase2. public class ApiDemosRobotiumTest extends ActivityInstrumentationTestCase2<ApiDemos> { ... public ApiDemosRobotiumTest() { super(ApiDemos.class); } ...
Controlling UI Using Robotium 4. Create an instance of Solo in the setUp method and close open activities in the tearDown method. ... private Solo solo; ... @Override public void setUp() throws Exception { solo = new Solo(getInstrumentation(), getActivity()); } @Override public void tearDown() throws Exception { solo.finishOpenedActivities(); } ...
Controlling UI Using Robotium 4. Find and act on a control. public void testNavigate() { solo.assertCurrentActivity("Should be on ApiDemos", ApiDemos.class); solo.clickOnText("^Graphics$", 1, true); solo.assertCurrentActivity("Should be on a new ApiDemos", ApiDemos.class, true); solo.clickOnText("^OpenGL ES$", 1, true); solo.assertCurrentActivity("Should be on a new ApiDemos", ApiDemos.class, true); solo.clickOnText("^Kube$", 1, true); solo.assertCurrentActivity("Should be on a Kube", Kube.class, true); solo.sleep(3000); }
Controlling UI Using Robotium 6. Build and Install the test package. $ cd ApiDemosRobotiumTest $ ant debug $ adb install –r bin/ApiDemosRobotiumTest-debug.apk
Controlling UI Using Robotium 6. Build and Install the test package. $ cd ApiDemosRobotiumTest $ ant debug $ adb install –r bin/ApiDemosRobotiumTest-debug.apk 7. Run the test and see the result. $ adb shell am instrument -w > com.example.android.apis.robotium.test/ > android.test.InstrumentationTestRunner
Controlling UI Using Robotium 6. Build and Install the test package. $ cd ApiDemosRobotiumTest $ ant debug $ adb install –r bin/ApiDemosRobotiumTest-debug.apk 7. Run the test and see the result. $ adb shell am instrument -w > com.example.android.apis.robotium.test/ > android.test.InstrumentationTestRunner $ ant test
Controlling UI Using uiautomator 1. Create an Java Project for the test jar.
Controlling UI Using uiautomator 1. Create an Java Project for the test jar. 2. Add ‘android.jar’ and ‘uiautomator.jar’ in the build path.
Controlling UI Using uiautomator 1. Create an Java Project for the test jar. 2. Add ‘android.jar’ and ‘uiautomator.jar’ in the build path. 3. Add a test class inheriting UiAutomatorTestCase. public class ApiDemosUiAutoTest extends UiAutomatorTestCase { ... }
Controlling UI Using uiautomator 4. Launch ApiDemos. public void testNavigate() throws UiObjectNotFoundException {
Controlling UI Using uiautomator 4. Launch ApiDemos. public void testNavigate() throws UiObjectNotFoundException { getUiDevice().pressHome(); Press home button
Controlling UI Using uiautomator 4. Launch ApiDemos. public void testNavigate() throws UiObjectNotFoundException { getUiDevice().pressHome(); UiObject allAppsButton = new UiObject( new UiSelector().description("Apps")); allAppsButton.clickAndWaitForNewWindow(); Press home button Click ‘All Apps’ button
Controlling UI Using uiautomator 4. Launch ApiDemos. public void testNavigate() throws UiObjectNotFoundException { getUiDevice().pressHome(); UiObject allAppsButton = new UiObject( new UiSelector().description("Apps")); allAppsButton.clickAndWaitForNewWindow(); Press home button Click ‘All Apps’ button UiObject appsTab = new UiObject(new UiSelector().text("Apps")); appsTab.click(); Click ‘Apps’ tab
Controlling UI Using uiautomator 4. Launch ApiDemos. public void testNavigate() throws UiObjectNotFoundException { getUiDevice().pressHome(); UiObject allAppsButton = new UiObject( new UiSelector().description("Apps")); allAppsButton.clickAndWaitForNewWindow(); Press home button Click ‘All Apps’ button UiObject appsTab = new UiObject(new UiSelector().text("Apps")); appsTab.click(); Click ‘Apps’ tab UiScrollable appList = new UiScrollable( new UiSelector().scrollable(true)); appList.setAsHorizontalList(); UiObject apiDemos = appList.getChildByText( new UiSelector().className(TextView.class.getName()), "API Demos"); apiDemos.clickAndWaitForNewWindow(); Click ‘ApiDemos’ icon
Controlling UI Using uiautomator 4. Launch ApiDemos. public void testNavigate() throws UiObjectNotFoundException { getUiDevice().pressHome(); UiObject allAppsButton = new UiObject( new UiSelector().description("Apps")); allAppsButton.clickAndWaitForNewWindow(); Press home button Click ‘All Apps’ button UiObject appsTab = new UiObject(new UiSelector().text("Apps")); appsTab.click(); Click ‘Apps’ tab UiScrollable appList = new UiScrollable( new UiSelector().scrollable(true)); appList.setAsHorizontalList(); UiObject apiDemos = appList.getChildByText( new UiSelector().className(TextView.class.getName()), "API Demos"); apiDemos.clickAndWaitForNewWindow(); Click ‘ApiDemos’ icon Validate the launch of ‘ApiDemos’ UiObject apiDemosPackage = new UiObject( new UiSelector().packageName("com.example.android.apis")); assertTrue(“Should be on ApiDemos", apiDemosPackage.exists()); ...
Controlling UI Using uiautomator 5. Find and act on a control.
Controlling UI Using uiautomator 5. Find and act on a control. ... UiScrollable demoList = new UiScrollable( new UiSelector().className("android.widget.ListView"));
Controlling UI Using uiautomator 5. Find and act on a control. ... UiScrollable demoList = new UiScrollable( new UiSelector().className("android.widget.ListView")); UiObject graphicsDemo = demoList.getChildByText( new UiSelector().className(TextView.class.getName()), “Graphics");
Controlling UI Using uiautomator 5. Find and act on a control. ... UiScrollable demoList = new UiScrollable( new UiSelector().className("android.widget.ListView")); UiObject graphicsDemo = demoList.getChildByText( new UiSelector().className(TextView.class.getName()), “Graphics"); graphicsDemo.clickAndWaitForNewWindow();
Controlling UI Using uiautomator 5. Find and act on a control. ... UiScrollable demoList = new UiScrollable( new UiSelector().className("android.widget.ListView")); UiObject graphicsDemo = demoList.getChildByText( new UiSelector().className(TextView.class.getName()), “Graphics"); graphicsDemo.clickAndWaitForNewWindow(); UiObject alphaBitmapDemo = new UiObject( new UiSelector().text(“AlphaBitmap")); assertTrue(“AlphaBitmap should be visible", alpahBitmapDemo.exists()); ...
Controlling UI Using uiautomator 5. Find and act on a control. ... UiScrollable demoList = new UiScrollable( new UiSelector().className("android.widget.ListView")); UiObject graphicsDemo = demoList.getChildByText( new UiSelector().className(TextView.class.getName()), “Graphics"); graphicsDemo.clickAndWaitForNewWindow(); UiObject alphaBitmapDemo = new UiObject( new UiSelector().text(“AlphaBitmap")); assertTrue(“AlphaBitmap should be visible", alpahBitmapDemo.exists()); ... 6. Iterate step 5.
Controlling UI Using uiautomator 7. Compile your test case into a JAR file. $ > $ $ android create uitest-project --name ApiDemosUiAutoTest --target 1 --path ApiDemosUiAutoTest cd ApiDemosUiAutoTest ant build
Controlling UI Using uiautomator 7. Compile your test case into a JAR file. $ > $ $ android create uitest-project --name ApiDemosUiAutoTest --target 1 --path ApiDemosUiAutoTest cd ApiDemosUiAutoTest ant build 8. Install the jar on your test device. $ adb push bin/ApiDemosUiAutoTest.jar /data/local/tmp/
Controlling UI Using uiautomator 7. Compile your test case into a JAR file. $ > $ $ android create uitest-project --name ApiDemosUiAutoTest --target 1 --path ApiDemosUiAutoTest cd ApiDemosUiAutoTest ant build 8. Install the jar on your test device. $ adb push bin/ApiDemosUiAutoTest.jar /data/local/tmp/ 9. Run the test and view the test result. $ adb shell uiautomator runtest ApiDemosUiAutoTest.jar > –c com.example.android.apis.uiauto.test.ApiDemosUiAutoTest
Controlling UI Using uiautomator 7. Compile your test case into a JAR file. $ > $ $ android create uitest-project --name ApiDemosUiAutoTest --target 1 --path ApiDemosUiAutoTest cd ApiDemosUiAutoTest ant build 8. Install the jar on your test device. $ adb push bin/ApiDemosUiAutoTest.jar /data/local/tmp/ 9. Run the test and view the test result. $ adb shell uiautomator runtest ApiDemosUiAutoTest.jar > –c com.example.android.apis.uiauto.test.ApiDemosUiAutoTest $ ant test
Controlling UI Using uiautomator 7. Compile your test case into a JAR file. $ > $ $ android create uitest-project --name ApiDemosUiAutoTest --target 1 --path ApiDemosUiAutoTest cd ApiDemosUiAutoTest ant build 8. Install the jar on your test device. $ adb push bin/ApiDemosUiAutoTest.jar /data/local/tmp/ 9. Run the test and view the test result. $ adb shell uiautomator runtest ApiDemosUiAutoTest.jar > –c com.example.android.apis.uiauto.test.ApiDemosUiAutoTest $ ant test
Doing both Controlling and Tracking
Doing both Controlling and Tracking 1. Modify org.kandroid.memtracer.MemoryInstrumentation. public class MemoryInstrumentation extends InstrumentationTestRunner { ... @Override public void onCreate(Bundle arguments) { ... start(); super.onCreate(arguments); } @Override public void onStart() { String mainActivityName = getMainActivityName(); if (mainActivityName != null && mainActivityName.length() > 0) { launchMainActivity(getTargetContext().getPackageName(), getMainActivityName()); } else { super.onStart(); } } ...
Doing both Controlling and Tracking 1. Modify org.kandroid.memtracer.MemoryInstrumentation. public class MemoryInstrumentation extends InstrumentationTestRunner { ... @Override public void onCreate(Bundle arguments) { ... start(); super.onCreate(arguments); } @Override public void onStart() { String mainActivityName = getMainActivityName(); if (mainActivityName != null && mainActivityName.length() > 0) { launchMainActivity(getTargetContext().getPackageName(), getMainActivityName()); } else { super.onStart(); } } ...
Doing both Controlling and Tracking 1. Modify org.kandroid.memtracer.MemoryInstrumentation. public class MemoryInstrumentation extends InstrumentationTestRunner { ... @Override public void onCreate(Bundle arguments) { ... start(); super.onCreate(arguments); } @Override public void onStart() { String mainActivityName = getMainActivityName(); if (mainActivityName != null && mainActivityName.length() > 0) { launchMainActivity(getTargetContext().getPackageName(), getMainActivityName()); } else { super.onStart(); } } ...
Doing both Controlling and Tracking 1. Modify org.kandroid.memtracer.MemoryInstrumentation. public class MemoryInstrumentation extends InstrumentationTestRunner { ... @Override public void onCreate(Bundle arguments) { ... start(); super.onCreate(arguments); } @Override public void onStart() { String mainActivityName = getMainActivityName(); if (mainActivityName != null && mainActivityName.length() > 0) { launchMainActivity(getTargetContext().getPackageName(), getMainActivityName()); } else { super.onStart(); } } ...
Doing both Controlling and Tracking 2. Add ‘kmemtracer-libs.jar’ in the ‘libs’ directory.
Doing both Controlling and Tracking 2. Add ‘kmemtracer-libs.jar’ in the ‘libs’ directory. 3. Edit the <instrumentation> element in the manifest file.
Doing both Controlling and Tracking 2. Add ‘kmemtracer-libs.jar’ in the ‘libs’ directory. 3. Edit the <instrumentation> element in the manifest file. 4. Build and Install the test package. $ cd ApiDemosInstrumentTest $ ant debug $ adb install –r bin/ApiDemosInstrumentTest-debug.apk
Doing both Controlling and Tracking 2. Add ‘kmemtracer-libs.jar’ in the ‘libs’ directory. 3. Edit the <instrumentation> element in the manifest file. 4. Build and Install the test package. $ cd ApiDemosInstrumentTest $ ant debug $ adb install –r bin/ApiDemosInstrumentTest-debug.apk 5. Run the test and see the result. $ adb shell am instrument -w > com.example.android.apis.instrument.test/ > org.kandroid.memtracer.MemoryInstrumentation
Doing both Controlling and Tracking 2. Add ‘kmemtracer-libs.jar’ in the ‘libs’ directory. 3. Edit the <instrumentation> element in the manifest file. 4. Build and Install the test package. $ cd ApiDemosInstrumentTest $ ant debug $ adb install –r bin/ApiDemosInstrumentTest-debug.apk 5. Run the test and see the result. $ adb shell am instrument -w > com.example.android.apis.instrument.test/ > org.kandroid.memtracer.MemoryInstrumentation 6. Pull the trace file ‘kmemtrace.csv’ from /sdcard/kmemtracer. $ adb pull /sdcard/kmemtracer/kmemtrace.csv .
SUMMARY
SUMMARY Why Automated Performance Testing? Measure to improve / Fragmentation
SUMMARY Why Automated Performance Testing? Measure to improve / Fragmentation How to Automate Performance Testing? - Historical Analysis w/ kmemtracer Instrumentation for track
SUMMARY Why Automated Performance Testing? Measure to improve / Fragmentation How to Automate Performance Testing? - Historical Analysis w/ kmemtracer Instrumentation for track - Measuring Performance API (Public / Internal) / cmds (Android / Linux)
SUMMARY Why Automated Performance Testing? Measure to improve / Fragmentation How to Automate Performance Testing? - Historical Analysis w/ kmemtracer Instrumentation for track - Measuring Performance API (Public / Internal) / cmds (Android / Linux) - Controlling UI Instrumentation / Robotium / uiautomator
Q&A
THANK YOU

More Related Content

ODP
Unit Test Android Without Going Bald
David Carver
 
PDF
How to setup unit testing in Android Studio
tobiaspreuss
 
PDF
20150319 testotipsio
Kazuaki Matsuo
 
PPTX
BDD using Cucumber JVM
Vijay Krishnan Ramaswamy
 
PPTX
Optimizing apps for better performance extended
Elif Boncuk
 
PDF
2012 04-19 theory-of_operation
bobwolff68
 
PDF
Groovy Testing Aug2009
guest4a266c
 
PDF
Mastering the NDK with Android Studio 2.0 and the gradle-experimental plugin
Xavier Hallade
 
Unit Test Android Without Going Bald
David Carver
 
How to setup unit testing in Android Studio
tobiaspreuss
 
20150319 testotipsio
Kazuaki Matsuo
 
BDD using Cucumber JVM
Vijay Krishnan Ramaswamy
 
Optimizing apps for better performance extended
Elif Boncuk
 
2012 04-19 theory-of_operation
bobwolff68
 
Groovy Testing Aug2009
guest4a266c
 
Mastering the NDK with Android Studio 2.0 and the gradle-experimental plugin
Xavier Hallade
 

What's hot (20)

PPTX
Optimizing Apps for Better Performance
Elif Boncuk
 
PPTX
Apache Maven - eXo VN office presentation
Arnaud Héritier
 
PDF
Espresso
kanthivel
 
PDF
Night Watch with QA
Carsten Sandtner
 
PDF
Building testable chrome extensions
Seth McLaughlin
 
PPTX
Maven basics
Vijay Krishnan Ramaswamy
 
PPTX
Tech Talk #5 : Android Automation Test with Espresso - Trần Văn Toàn
Nexus FrontierTech
 
PDF
High Performance JavaScript 2011
Nicholas Zakas
 
PDF
Test automation - Building effective solutions
Artem Nagornyi
 
PPT
Selenium-Browser-Based-Automated-Testing-for-Grails-Apps
chrisb206 chrisb206
 
PDF
maven build certificaton
Vskills
 
PDF
Alpes Jug (29th March, 2010) - Apache Maven
Arnaud Héritier
 
PDF
Geneva Jug (30th March, 2010) - Maven
Arnaud Héritier
 
PDF
Tellurium At Rich Web Experience2009
John.Jian.Fang
 
PPTX
Jasmine with JS-Test-Driver
Devesh Chanchlani
 
PDF
Android App development and test environment, Understaing android app structure
Vijay Rastogi
 
PDF
Why gradle
Sercan Karaoglu
 
PDF
Android on IA devices and Intel Tools
Xavier Hallade
 
PDF
Google App Engine for Java v0.0.2
Matthew McCullough
 
PDF
Testing With Open Source
Matthias Käppler
 
Optimizing Apps for Better Performance
Elif Boncuk
 
Apache Maven - eXo VN office presentation
Arnaud Héritier
 
Espresso
kanthivel
 
Night Watch with QA
Carsten Sandtner
 
Building testable chrome extensions
Seth McLaughlin
 
Maven basics
Vijay Krishnan Ramaswamy
 
Tech Talk #5 : Android Automation Test with Espresso - Trần Văn Toàn
Nexus FrontierTech
 
High Performance JavaScript 2011
Nicholas Zakas
 
Test automation - Building effective solutions
Artem Nagornyi
 
Selenium-Browser-Based-Automated-Testing-for-Grails-Apps
chrisb206 chrisb206
 
maven build certificaton
Vskills
 
Alpes Jug (29th March, 2010) - Apache Maven
Arnaud Héritier
 
Geneva Jug (30th March, 2010) - Maven
Arnaud Héritier
 
Tellurium At Rich Web Experience2009
John.Jian.Fang
 
Jasmine with JS-Test-Driver
Devesh Chanchlani
 
Android App development and test environment, Understaing android app structure
Vijay Rastogi
 
Why gradle
Sercan Karaoglu
 
Android on IA devices and Intel Tools
Xavier Hallade
 
Google App Engine for Java v0.0.2
Matthew McCullough
 
Testing With Open Source
Matthias Käppler
 
Ad

Similar to DEVIEW2013: Automating Performance Tests for Android Applications (20)

PDF
Automated Historical Performance Analysis with kmemtracer
Kyungmin Lee
 
PPTX
Of Bytes, Cycles and Battery Life
Mario Zechner
 
PDF
YuryMakedonov_TesTrek2013_AndroidTesting_12u_slides
Yury M
 
PDF
Mobile Saturday. Тема 2. Особенности тестирования приложения на Android: Spec...
GoIT
 
PDF
Unit 1 Tools Beneficial for Monitoring the Debugging Process.pdf
ChatanBawankar
 
PPTX
Android Mobile Application Testing: Specific Functional, Performance, Device ...
SoftServe
 
PDF
Profiling Android Applications
hubx
 
PPTX
Londroid Akshay Performance
Skills Matter
 
PPTX
Performance Testing on Android
Akshay Dashrath
 
PPT
Android developmenttools 20100424
Marakana Inc.
 
PDF
Mobile Day - Intel XDK & Testing
Software Guru
 
PDF
Desarrollo y testing de apps móviles con Intel XDK y Testdroid
Software Guru
 
PPTX
Android tools for testers
Maksim Kovalev
 
PDF
Andriod dev toolbox part 2
Shem Magnezi
 
PPTX
Instrumentation 101
Apkudo
 
DOCX
UIAutomator
Sandip Ganguli
 
PPTX
Android testing
Bitbar
 
PDF
MobSecCon 2015 - Dynamic Analysis of Android Apps
Ron Munitz
 
PDF
Android Boot Time Optimization
Kan-Ru Chen
 
PPTX
Performance and Power Profiling on Intel Android Devices
Intel® Software
 
Automated Historical Performance Analysis with kmemtracer
Kyungmin Lee
 
Of Bytes, Cycles and Battery Life
Mario Zechner
 
YuryMakedonov_TesTrek2013_AndroidTesting_12u_slides
Yury M
 
Mobile Saturday. Тема 2. Особенности тестирования приложения на Android: Spec...
GoIT
 
Unit 1 Tools Beneficial for Monitoring the Debugging Process.pdf
ChatanBawankar
 
Android Mobile Application Testing: Specific Functional, Performance, Device ...
SoftServe
 
Profiling Android Applications
hubx
 
Londroid Akshay Performance
Skills Matter
 
Performance Testing on Android
Akshay Dashrath
 
Android developmenttools 20100424
Marakana Inc.
 
Mobile Day - Intel XDK & Testing
Software Guru
 
Desarrollo y testing de apps móviles con Intel XDK y Testdroid
Software Guru
 
Android tools for testers
Maksim Kovalev
 
Andriod dev toolbox part 2
Shem Magnezi
 
Instrumentation 101
Apkudo
 
UIAutomator
Sandip Ganguli
 
Android testing
Bitbar
 
MobSecCon 2015 - Dynamic Analysis of Android Apps
Ron Munitz
 
Android Boot Time Optimization
Kan-Ru Chen
 
Performance and Power Profiling on Intel Android Devices
Intel® Software
 
Ad

Recently uploaded (20)

PPTX
sap open course for s4hana steps from ECC to s4
sreeni2106invites
 
PDF
Encapsulation theory and applications.pdf
gurumoop
 
PDF
MIND Revenue Release Quarter 2 2025 Press Release
MIND CTI
 
PDF
NewMind AI Weekly Chronicles - August'25 Week I
NewMind AI
 
PDF
Mobile App Security Testing_ A Comprehensive Guide.pdf
flufftailshop
 
PDF
Empathic Computing: Creating Shared Understanding
Mark Billinghurst
 
PDF
Diabetes mellitus diagnosis method based random forest with bat algorithm
IAESIJAI
 
PPTX
20250228 LYD VKU AI Blended-Learning.pptx
DatVoNgoc
 
PDF
Network Security Unit 5.pdf for BCA BBA.
Serpent6
 
PDF
Agricultural_Statistics_at_a_Glance_2022_0.pdf
SandeepSingh286037
 
PPTX
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
ReZa AdineH
 
PDF
cuic standard and advanced reporting.pdf
SimoneTitaniumTomase
 
PPTX
Programs and apps: productivity, graphics, security and other tools
4mqw9zch22
 
PDF
Approach and Philosophy of On baking technology
30anthuong17
 
PDF
Dropbox Q2 2025 Financial Results & Investor Presentation
Dropbox
 
PDF
Profit Center Accounting in SAP S/4HANA, S4F28 Col11
Libreria ERP
 
PPTX
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
blackmambaettijean
 
PDF
Review of recent advances in non-invasive hemoglobin estimation
IAESIJAI
 
PDF
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
Akar16
 
PDF
TokAI - TikTok AI Agent : The First AI Application That Analyzes 10,000+ Vira...
SOFTTECHHUB
 
sap open course for s4hana steps from ECC to s4
sreeni2106invites
 
Encapsulation theory and applications.pdf
gurumoop
 
MIND Revenue Release Quarter 2 2025 Press Release
MIND CTI
 
NewMind AI Weekly Chronicles - August'25 Week I
NewMind AI
 
Mobile App Security Testing_ A Comprehensive Guide.pdf
flufftailshop
 
Empathic Computing: Creating Shared Understanding
Mark Billinghurst
 
Diabetes mellitus diagnosis method based random forest with bat algorithm
IAESIJAI
 
20250228 LYD VKU AI Blended-Learning.pptx
DatVoNgoc
 
Network Security Unit 5.pdf for BCA BBA.
Serpent6
 
Agricultural_Statistics_at_a_Glance_2022_0.pdf
SandeepSingh286037
 
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
ReZa AdineH
 
cuic standard and advanced reporting.pdf
SimoneTitaniumTomase
 
Programs and apps: productivity, graphics, security and other tools
4mqw9zch22
 
Approach and Philosophy of On baking technology
30anthuong17
 
Dropbox Q2 2025 Financial Results & Investor Presentation
Dropbox
 
Profit Center Accounting in SAP S/4HANA, S4F28 Col11
Libreria ERP
 
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
blackmambaettijean
 
Review of recent advances in non-invasive hemoglobin estimation
IAESIJAI
 
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
Akar16
 
TokAI - TikTok AI Agent : The First AI Application That Analyzes 10,000+ Vira...
SOFTTECHHUB
 

DEVIEW2013: Automating Performance Tests for Android Applications

  • 1. 안드로이드 앱의 성능 테스트, 자동화가 가능할까? 이경민 / LG전자
  • 10. CONTENTS Why Automated Performance Testing? How to Automate Performance Testing? - Historical Analysis w/ kmemtracer - Measuring Performance - Controlling UI
  • 12. If you can’t measure it, you can’t manage it. Peter F. Drucker (1909~2005)
  • 13. If you can’t measure it, you can’t manage it. Peter F. Drucker (1909~2005) If you can’t automate the measurement of it, it is difficult to improve it.
  • 15. Diversity or Fragmentation Platform Versions Screen Sizes and Densities http://developer.android.com/about/dashboards/index.html
  • 17. How to Automate Performance Testing? - Historical Analysis w/ kmemtracer - Measuring Performance - Controlling UI
  • 18. Why Historical Analysis? Source: Memory and Performance at the 11th Korea Android Conference
  • 19. Why Historical Analysis? Source: Memory and Performance at the 11th Korea Android Conference
  • 20. Why Historical Analysis? Activity Navigation & Lifecycle Source: Memory and Performance at the 11th Korea Android Conference
  • 21. How to Track Activity Navigation & Lifecycle?
  • 22. How to Track Activity Navigation & Lifecycle? Override onCreate, onStart, … of your Activity classes
  • 23. How to Track Activity Navigation & Lifecycle? Modify android.app.Activity in the framework
  • 24. How to Track Activity Navigation & Lifecycle? Is there a way that does not require the modification of an application or framework?
  • 25. How to Track Activity Navigation & Lifecycle? Is there a way that does not require the modification of an application or framework? Instrumentation
  • 27. Android instrumentation is a set of control methods or "hooks" in the Android system. These hooks control an Android component independently of its normal lifecycle. They also control how Android loads applications. (http://developer.android.com/tools/testing/testing_android.html)
  • 28. Historical Analysis w/ kmemtracer Application Package Control android.test. Instrumentation TestRunner Test Package
  • 29. Historical Analysis w/ kmemtracer https://github.com/snailee/kmemtracer-libs Application Package Control Track android.test. Instrumentation TestRunner org.kandroid. memtracer.Memory Instrumentation Test Package Trace Package
  • 30. android.app.Instrumentation void void void void void void void void void void void void void callActivityOnCreate(Activity, Bundle) callActivityOnDestroy(Activity) callActivityOnNewIntent(Activity, Intent) callActivityOnPause(Activity) callActivityOnPostCreate(Activity, Bundle) callActivityOnRestart(Activity) callActivityOnRestoreInstanceState(Activity, Bundle) callActivityOnResume(Activity) callActivityOnSaveInstanceState(Activity, Bundle) callActivityOnStart(Activity) callActivityOnStop(Activity) callActivityOnUserLeaving(Activity) callApplicationOnCreate(Application)
  • 31. android.app.Instrumentation void void void void void void void void void void void void void callActivityOnCreate(Activity, Bundle) callActivityOnDestroy(Activity) callActivityOnNewIntent(Activity, Intent) callActivityOnPause(Activity) callActivityOnPostCreate(Activity, Bundle) callActivityOnRestart(Activity) callActivityOnRestoreInstanceState(Activity, Bundle) callActivityOnResume(Activity) callActivityOnSaveInstanceState(Activity, Bundle) callActivityOnStart(Activity) callActivityOnStop(Activity) callActivityOnUserLeaving(Activity) callApplicationOnCreate(Application) Call for Control
  • 32. android.app.Instrumentation void void void void void void void void void void void void void callActivityOnCreate(Activity, Bundle) callActivityOnDestroy(Activity) callActivityOnNewIntent(Activity, Intent) callActivityOnPause(Activity) callActivityOnPostCreate(Activity, Bundle) callActivityOnRestart(Activity) callActivityOnRestoreInstanceState(Activity, Bundle) callActivityOnResume(Activity) callActivityOnSaveInstanceState(Activity, Bundle) callActivityOnStart(Activity) callActivityOnStop(Activity) callActivityOnUserLeaving(Activity) callApplicationOnCreate(Application) Call for Control Override for Track
  • 33. org.kandroid.memtracer.MemoryInstrumentation public class MemoryInstrumentation extends Instrumentation { private static final String ARGUMENT_INSTRUMENT_CLASS = "class"; private MemoryTracer mMemoryTracer; private String mMainActivityName; ... @Override public void onCreate(Bundle arguments) { mMainActivityName = arguments.getString(ARGUMENT_INSTRUMENT_CLASS); mMemoryTracer = createMemoryTracer(); mMemoryTracer.startTracing(getTargetContext().getPackageName()); start(); } @Override public void callActivityOnCreate(Activity activity, Bundle icicle) { String tag = activity.getLocalClassName()+"-OnCreate"; Bundle snapshot = mMemoryTracer.addSnapshot(tag); super.callActivityOnCreate(activity, icicle); }
  • 34. How to Use kmemtracer?
  • 35. How to Use kmemtracer? 1. Create an Android Test Project for the trace package.
  • 36. How to Use kmemtracer? 1. Create an Android Test Project for the trace package. 2. Add ‘kmemtracer-libs.jar’ in the ‘libs’ directory.
  • 37. How to Use kmemtracer? 1. Create an Android Test Project for the trace package. 2. Add ‘kmemtracer-libs.jar’ in the ‘libs’ directory. 3. Edit the <instrumentation> element in the Manifest file. <manifest xmlns:android="http://schemas.android.com/apk/res/android" package="com.example.android.apis.test" android:versionCode="1" android:versionName="1.0"> <instrumentation android:name="org.kandroid.memtracer.MemoryInstrumentation" android:targetPackage="com.example.android.apis" /> <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"/> <application android:label="@string/app_name" android:icon="@drawable/ic_launcher“> <uses-library android:theme="android.text.runner“> </application> </manifest>
  • 38. How to Use kmemtracer? 4. Install the trace package as well as the application package. $ adb install –r ApiDemos.apk $ adb install –r ApiDemosTest.apk
  • 39. How to Use kmemtracer? 4. Install the trace package as well as the application package. $ adb install –r ApiDemos.apk $ adb install –r ApiDemosTest.apk 5. Start the instrumentation with ‘am instrument’ in the shell. $ > > > adb shell am instrument -e class com.example.android.apis.ApiDemos com.example.android.apis.tests/ org.kandroid.memtracer.MemoryInstrumentation
  • 40. How to Use kmemtracer? 4. Install the trace package as well as the application package. $ adb install –r ApiDemos.apk $ adb install –r ApiDemosTest.apk 5. Start the instrumentation with ‘am instrument’ in the shell. $ > > > adb shell am instrument -e class com.example.android.apis.ApiDemos com.example.android.apis.tests/ org.kandroid.memtracer.MemoryInstrumentation 6. Do interact with the application.
  • 41. How to Use kmemtracer? 4. Install the trace package as well as the application package. $ adb install –r ApiDemos.apk $ adb install –r ApiDemosTest.apk 5. Start the instrumentation with ‘am instrument’ in the shell. $ > > > adb shell am instrument -e class com.example.android.apis.ApiDemos com.example.android.apis.tests/ org.kandroid.memtracer.MemoryInstrumentation 6. Do interact with the application. 7. Pull the trace file ‘kmemtrace.csv’ from /sdcard/kmemtracer. $ adb pull /sdcard/kmemtracer/kmemtrace.csv .
  • 42. How to Use kmemtracer? 8. Open the trace file with Excel and create charts.
  • 43. How to Automate Performance Testing? - Historical Analysis w/ kmemtracer - Measuring Performance - Controlling UI
  • 44. What to Measure? Rendering (or GUI) Performance Frame Rate (FPS), Frame Time , Jankiness, Input Latency, … Resource Consumption Memory, CPU, I/O (Disk, Network), Battery, … Network Performance Response time, Throughput, … …
  • 45. What to Measure? Rendering (or GUI) Performance Frame Rate (FPS), Frame Time , Jankiness, Input Latency, … Resource Consumption Memory, CPU, I/O (Disk, Network), Battery, … Network Performance Response time, Throughput, … …
  • 46. How to Measure Resource Consumption?
  • 47. How to Measure Resource Consumption? API cmds
  • 48. How to Measure Resource Consumption? cmds API Public Internal
  • 49. How to Measure Resource Consumption? cmds API Public Internal Android Linux
  • 50. How to Measure Resource Consumption? cmds API Public Easy Internal Android Linux Flexible
  • 55. Measuring Memory Usage Public Internal Android Linux android.app.ActivityManager.MemoryInfo android.os.Debug android.os.Debug.MemoryInfo dalvik.system.VMDebug adb shell dumpsys meminfo [pid|pname] adb shell procrank adb shell cat /proc/meminfo adb shell cat /proc/<pid>/[s]maps adb shell cat /proc/vmstat adb shell top adb shell ps
  • 71. Tracing Performance Snapshots in kmemtracer public class MemoryInstrumentation extends Instrumentation {
  • 72. Tracing Performance Snapshots in kmemtracer public class MemoryInstrumentation extends Instrumentation { ... private MemoryTracer mMemoryTracer; ... @Override public void onCreate(Bundle arguments) { ... mMemoryTracer = createMemoryTracer(); ... } ... Create an instance of MemoryTracer
  • 73. Tracing Performance Snapshots in kmemtracer public class MemoryInstrumentation extends Instrumentation { ... private MemoryTracer mMemoryTracer; ... @Override public void onCreate(Bundle arguments) { ... mMemoryTracer = createMemoryTracer(); ... } ... @Override public void callActivityOnCreate(Activity activity, Bundle icicle) { String tag = activity.getLocalClassName()+"-OnCreate"; Bundle snapshot = mMemoryTracer.addSnapshot(tag); ... } ... } Create an instance of MemoryTracer Add performance snapshot
  • 75. MemoryTracer$addSnapshot public Bundle addSnapshot(String label) { ... long nativeMax = Debug.getNativeHeapSize() / 1024; long nativeAllocated = Debug.getNativeHeapAllocatedSize() / 1024; long nativeFree = Debug.getNativeHeapFreeSize() / 1024; Debug.MemoryInfo memInfo = new Debug.MemoryInfo(); Debug.getMemoryInfo(memInfo); ... Get usage data
  • 76. MemoryTracer$addSnapshot public Bundle addSnapshot(String label) { ... long nativeMax = Debug.getNativeHeapSize() / 1024; long nativeAllocated = Debug.getNativeHeapAllocatedSize() / 1024; long nativeFree = Debug.getNativeHeapFreeSize() / 1024; Debug.MemoryInfo memInfo = new Debug.MemoryInfo(); Debug.getMemoryInfo(memInfo); ... Bundle snapshot = new Bundle(); snapshot.putString(METRIC_KEY_LABEL, label); ... snapshot.putLong(METRIC_KEY_NATIVE_SIZE, nativeMax); snapshot.putLong(METRIC_KEY_NATIVE_ALLOCATED, nativeAllocated); snapshot.putLong(METRIC_KEY_NATIVE_FREE, nativeFree); ... Get usage data Save data to a bundle
  • 77. MemoryTracer$addSnapshot public Bundle addSnapshot(String label) { ... long nativeMax = Debug.getNativeHeapSize() / 1024; long nativeAllocated = Debug.getNativeHeapAllocatedSize() / 1024; long nativeFree = Debug.getNativeHeapFreeSize() / 1024; Debug.MemoryInfo memInfo = new Debug.MemoryInfo(); Debug.getMemoryInfo(memInfo); ... Bundle snapshot = new Bundle(); snapshot.putString(METRIC_KEY_LABEL, label); ... snapshot.putLong(METRIC_KEY_NATIVE_SIZE, nativeMax); snapshot.putLong(METRIC_KEY_NATIVE_ALLOCATED, nativeAllocated); snapshot.putLong(METRIC_KEY_NATIVE_FREE, nativeFree); ... if (mResultsWriter != null) { mResultsWriter.writeTraceSnapshot(snapshot); } return snapshot; } Get usage data Save data to a bundle Write date using ResultsWriter
  • 79. MemoryTracerCsvWriter$writeTraceSnapshot public class MemoryTraceCsvWriter implements MemoryTracer.ResultsWriter { ... public MemoryTraceCsvWriter(String[] metricKeys) { mMetricKeys = metricKeys; } ... Create an instance with metrics to print
  • 80. MemoryTracerCsvWriter$writeTraceSnapshot public class MemoryTraceCsvWriter implements MemoryTracer.ResultsWriter { ... public MemoryTraceCsvWriter(String[] metricKeys) { mMetricKeys = metricKeys; } ... private void openTraceFile(String filename) { ... mTraceOut = new PrintStream(new FileOutputStream(traceFile)); ... } ... Create an instance with metrics to print Open a trace file in the sdcard at init
  • 81. MemoryTracerCsvWriter$writeTraceSnapshot public class MemoryTraceCsvWriter implements MemoryTracer.ResultsWriter { ... public MemoryTraceCsvWriter(String[] metricKeys) { mMetricKeys = metricKeys; } ... private void openTraceFile(String filename) { ... mTraceOut = new PrintStream(new FileOutputStream(traceFile)); ... } ... @Override public void writeTraceSnapshot(Bundle snapshot) { PrintStream out = mTraceOut; for (String key : mMetricKeys) { out.print(snapshot.get(key)); out.print(','); } out.println(); out.flush(); } ... } Create an instance with metrics to print Open a trace file in the sdcard at init Write snapshot to a line of the file
  • 83. To Do’s Add more tracers CpuTracer IOTracer BatteryTracer
  • 84. To Do’s Add more tracers CpuTracer IOTracer BatteryTracer Make metrics to gather configurable
  • 85. To Do’s Add more tracers CpuTracer IOTracer BatteryTracer Make metrics to gather configurable Make trace output destination configurable Trace out to internal file (not on sdcard) Trace out to an analytics server
  • 86. To Do’s Add more tracers CpuTracer IOTracer BatteryTracer Make metrics to gather configurable Make trace output destination configurable Trace out to internal file (not on sdcard) Trace out to an analytics server Welcome contributions https://github.com/snailee/kmemtracer-libs
  • 87. How to Automate Performance Testing? - Historical Analysis w/ kmemtracer - Measuring Performance - Controlling UI
  • 92. Three Alternatives Instrumentation API Level 1 ~ Single activity Framework APIs White-box http://developer.android.com/tools/testing/testing_android.html
  • 93. Three Alternatives Instrumentation API Level 1 ~ Single activity Framework APIs White-box Robotium 2010/01 ~ Multiple activities High-level APIs White-box & black-box http://developer.android.com/tools/testing/testing_android.html https://code.google.com/p/robotium/
  • 94. Three Alternatives Instrumentation API Level 1 ~ Single activity Framework APIs White-box Robotium uiautomator 2010/01 ~ Multiple activities API Level 17 ~ High-level APIs White-box & black-box Multiple processes Simple APIs Black-box http://developer.android.com/tools/testing/testing_android.html https://code.google.com/p/robotium/ http://developer.android.com/tools/testing/testing_ui.html
  • 104. Controlling UI Using Instrumentation
  • 105. Controlling UI Using Instrumentation 1. Create an Android Test Project for the test package. $ android create test-project --main ../ApiDemos > –-name ApiDemosInstrumentTest --path ApiDemosInstrumentTest
  • 106. Controlling UI Using Instrumentation 1. Create an Android Test Project for the test package. $ android create test-project --main ../ApiDemos > –-name ApiDemosInstrumentTest --path ApiDemosInstrumentTest 2. Add a test class inheriting ActivityInstrumentationTestCase2. public class ApiDemosInstrumentTest extends ActivityInstrumentationTestCase2<ApiDemos> { ... public ApiDemosInstrumentTest() { super(ApiDemos.class); }
  • 107. Controlling UI Using Instrumentation 1. Create an Android Test Project for the test package. $ android create test-project --main ../ApiDemos > –-name ApiDemosInstrumentTest --path ApiDemosInstrumentTest 2. Add a test class inheriting ActivityInstrumentationTestCase2. public class ApiDemosInstrumentTest extends ActivityInstrumentationTestCase2<ApiDemos> { ... public ApiDemosInstrumentTest() { super(ApiDemos.class); } 3. Add a test method and launch the activity. public void testNavigate() { final ApiDemos apiDemos = getActivity(); ... }
  • 108. Controlling UI Using Instrumentation 4. Find and act on a control. public void testNavigate() {
  • 109. Controlling UI Using Instrumentation 4. Find and act on a control. Find a view to control public void testNavigate() { ... final ListView demoList = apiDemos.getListView(); final View graphicsDemo = getChildViewByText(demoList, "Graphics"); assertNotNull("Graphics demo should exist", graphicsDemo);
  • 110. Controlling UI Using Instrumentation 4. Find and act on a control. Find a view to control public void testNavigate() { ... final ListView demoList = apiDemos.getListView(); final View graphicsDemo = getChildViewByText(demoList, "Graphics"); assertNotNull("Graphics demo should exist", graphicsDemo); apiDemos.runOnUiThread(new Runnable() { public void run() { int pos = demoList.getPositionForView(graphicsDemo); demoList.smoothScrollToPosition(pos); demoList.setSelection(pos); demoList.performItemClick(graphicsDemo, pos, graphicsDemo.getId()); } }); ... Act on the view
  • 111. Controlling UI Using Instrumentation 4. Find and act on a control. Find a view to control public void testNavigate() { ... final ListView demoList = apiDemos.getListView(); final View graphicsDemo = getChildViewByText(demoList, "Graphics"); assertNotNull("Graphics demo should exist", graphicsDemo); apiDemos.runOnUiThread(new Runnable() { public void run() { int pos = demoList.getPositionForView(graphicsDemo); demoList.smoothScrollToPosition(pos); demoList.setSelection(pos); demoList.performItemClick(graphicsDemo, pos, graphicsDemo.getId()); } }); ... Act on the view
  • 112. Controlling UI Using Instrumentation 4. Find and act on a control: getChildViewByText. private View getChildViewByText(ListView listView, String text) {
  • 113. Controlling UI Using Instrumentation 4. Find and act on a control: getChildViewByText. private View getChildViewByText(ListView listView, String text) { View view = null; int count = listView.getCount(); Log.e(TAG, “no. children = " + count); for (int i = 0; i < count; i++) { TextView textView = (TextView) listView.getChildAt(i); Log.e(TAG, i + "-th text view is " + textView); Iterate through child views
  • 114. Controlling UI Using Instrumentation 4. Find and act on a control: getChildViewByText. private View getChildViewByText(ListView listView, String text) { View view = null; int count = listView.getCount(); Log.e(TAG, “no. children = " + count); for (int i = 0; i < count; i++) { TextView textView = (TextView) listView.getChildAt(i); Log.e(TAG, i + "-th text view is " + textView); if (textView != null && textView.getText().equals(text)) { view = textView; break; } Iterate through child views Compare texts
  • 115. Controlling UI Using Instrumentation 4. Find and act on a control: getChildViewByText. private View getChildViewByText(ListView listView, String text) { View view = null; int count = listView.getCount(); Log.e(TAG, “no. children = " + count); for (int i = 0; i < count; i++) { TextView textView = (TextView) listView.getChildAt(i); Log.e(TAG, i + "-th text view is " + textView); if (textView != null && textView.getText().equals(text)) { view = textView; break; } } return view; } Iterate through child views Compare texts
  • 116. Controlling UI Using Instrumentation 4. Find and act on a control: getChildViewByText. private View getChildViewByText(ListView listView, String text) { View view = null; int count = listView.getCount(); Log.e(TAG, “no. children = " + count); for (int i = 0; i < count; i++) { TextView textView = (TextView) listView.getChildAt(i); Log.e(TAG, i + "-th text view is " + textView); if (textView != null && textView.getText().equals(text)) { view = textView; break; } } return view; } com.example.android.apis.instrument.test.ApiDemosInstrumentTest: INSTRUMENTATION_RESULT: shortMsg=junit.framework.AssertionFailedError INSTRUMENTATION_RESULT: longMsg= junit.framework.AssertionFailedError: OpenGL ES demo should exist INSTRUMENTATION_CODE: 0 Iterate through child views Compare texts
  • 117. Controlling UI Using Instrumentation 4. Find and act on a control: Revised getChildViewByText. private View getChildViewByText(final ListView listView, String text) { ... for (int i = 0; i < count; i++) { TextView textView = (TextView) listView.getChildAt(i); if (textView != null && textView.getText().equals(text)) { view = textView; break; } if (textView == null) { final int lastPos = listView.getLastVisiblePosition(); getInstrumentation().runOnMainSync(new Runnable() { public void run() { listView.smoothScrollToPosition(lastPos + 5, lastPos); } }); getInstrumentation().waitForIdleSync(); i = listView.getFirstVisiblePosition(); } } return view; }
  • 118. Controlling UI Using Instrumentation 4. Find and act on a control: Revised getChildViewByText. private View getChildViewByText(final ListView listView, String text) { ... for (int i = 0; i < count; i++) { TextView textView = (TextView) listView.getChildAt(i); if (textView != null && textView.getText().equals(text)) { view = textView; break; } if (textView == null) { final int lastPos = listView.getLastVisiblePosition(); getInstrumentation().runOnMainSync(new Runnable() { public void run() { listView.smoothScrollToPosition(lastPos + 5, lastPos); } }); getInstrumentation().waitForIdleSync(); i = listView.getFirstVisiblePosition(); } } return view; } Scroll down
  • 119. Controlling UI Using Instrumentation 5. Monitor a new Activity to launch. public void testNavigate() { ... Instrumentation.ActivityMonitor graphicsDemoActivityMonitor = getInstrumentation().addMonitor( ApiDemos.class.getName(), null, false); final ApiDemos graphicsDemoActivity = (ApiDemos) graphicsDemoActivityMonitor.waitForActivity(); assertNotNull("Graphics Demo (ApiDemos) activity should not be null", graphicsDemoActivity); ...
  • 120. Controlling UI Using Instrumentation 5. Monitor a new Activity to launch. public void testNavigate() { ... Instrumentation.ActivityMonitor graphicsDemoActivityMonitor = getInstrumentation().addMonitor( ApiDemos.class.getName(), null, false); final ApiDemos graphicsDemoActivity = (ApiDemos) graphicsDemoActivityMonitor.waitForActivity(); assertNotNull("Graphics Demo (ApiDemos) activity should not be null", graphicsDemoActivity); ... 6. Iterate step 4 & 5.
  • 121. Controlling UI Using Instrumentation 7. Build and Install the test package. $ cd ApiDemosInstrumentTest $ ant debug $ adb install –r bin/ApiDemosInstrumentTest-debug.apk
  • 122. Controlling UI Using Instrumentation 7. Build and Install the test package. $ cd ApiDemosInstrumentTest $ ant debug $ adb install –r bin/ApiDemosInstrumentTest-debug.apk 8. Run the test and see the result. $ adb shell am instrument -w > com.example.android.apis.instrument.test/ > android.test.InstrumentationTestRunner
  • 123. Controlling UI Using Instrumentation 7. Build and Install the test package. $ cd ApiDemosInstrumentTest $ ant debug $ adb install –r bin/ApiDemosInstrumentTest-debug.apk 8. Run the test and see the result. $ adb shell am instrument -w > com.example.android.apis.instrument.test/ > android.test.InstrumentationTestRunner $ ant test
  • 124. Controlling UI Using Robotium
  • 125. Controlling UI Using Robotium 1. Create an Android Test Project for the test package. $ android create test-project --main ../ApiDemos > –-name ApiDemosRobotiumTest --path ApiDemosRobotiumTest
  • 126. Controlling UI Using Robotium 1. Create an Android Test Project for the test package. $ android create test-project --main ../ApiDemos > –-name ApiDemosRobotiumTest --path ApiDemosRobotiumTest 2. Add ‘robotium-solo-<ver>.jar’ in the ‘libs’ directory.
  • 127. Controlling UI Using Robotium 1. Create an Android Test Project for the test package. $ android create test-project --main ../ApiDemos > –-name ApiDemosRobotiumTest --path ApiDemosRobotiumTest 2. Add ‘robotium-solo-<ver>.jar’ in the ‘libs’ directory. 3. Add a test class inheriting ActivityInstrumentationTestCase2. public class ApiDemosRobotiumTest extends ActivityInstrumentationTestCase2<ApiDemos> { ... public ApiDemosRobotiumTest() { super(ApiDemos.class); } ...
  • 128. Controlling UI Using Robotium 4. Create an instance of Solo in the setUp method and close open activities in the tearDown method. ... private Solo solo; ... @Override public void setUp() throws Exception { solo = new Solo(getInstrumentation(), getActivity()); } @Override public void tearDown() throws Exception { solo.finishOpenedActivities(); } ...
  • 129. Controlling UI Using Robotium 4. Find and act on a control. public void testNavigate() { solo.assertCurrentActivity("Should be on ApiDemos", ApiDemos.class); solo.clickOnText("^Graphics$", 1, true); solo.assertCurrentActivity("Should be on a new ApiDemos", ApiDemos.class, true); solo.clickOnText("^OpenGL ES$", 1, true); solo.assertCurrentActivity("Should be on a new ApiDemos", ApiDemos.class, true); solo.clickOnText("^Kube$", 1, true); solo.assertCurrentActivity("Should be on a Kube", Kube.class, true); solo.sleep(3000); }
  • 130. Controlling UI Using Robotium 6. Build and Install the test package. $ cd ApiDemosRobotiumTest $ ant debug $ adb install –r bin/ApiDemosRobotiumTest-debug.apk
  • 131. Controlling UI Using Robotium 6. Build and Install the test package. $ cd ApiDemosRobotiumTest $ ant debug $ adb install –r bin/ApiDemosRobotiumTest-debug.apk 7. Run the test and see the result. $ adb shell am instrument -w > com.example.android.apis.robotium.test/ > android.test.InstrumentationTestRunner
  • 132. Controlling UI Using Robotium 6. Build and Install the test package. $ cd ApiDemosRobotiumTest $ ant debug $ adb install –r bin/ApiDemosRobotiumTest-debug.apk 7. Run the test and see the result. $ adb shell am instrument -w > com.example.android.apis.robotium.test/ > android.test.InstrumentationTestRunner $ ant test
  • 133. Controlling UI Using uiautomator 1. Create an Java Project for the test jar.
  • 134. Controlling UI Using uiautomator 1. Create an Java Project for the test jar. 2. Add ‘android.jar’ and ‘uiautomator.jar’ in the build path.
  • 135. Controlling UI Using uiautomator 1. Create an Java Project for the test jar. 2. Add ‘android.jar’ and ‘uiautomator.jar’ in the build path. 3. Add a test class inheriting UiAutomatorTestCase. public class ApiDemosUiAutoTest extends UiAutomatorTestCase { ... }
  • 136. Controlling UI Using uiautomator 4. Launch ApiDemos. public void testNavigate() throws UiObjectNotFoundException {
  • 137. Controlling UI Using uiautomator 4. Launch ApiDemos. public void testNavigate() throws UiObjectNotFoundException { getUiDevice().pressHome(); Press home button
  • 138. Controlling UI Using uiautomator 4. Launch ApiDemos. public void testNavigate() throws UiObjectNotFoundException { getUiDevice().pressHome(); UiObject allAppsButton = new UiObject( new UiSelector().description("Apps")); allAppsButton.clickAndWaitForNewWindow(); Press home button Click ‘All Apps’ button
  • 139. Controlling UI Using uiautomator 4. Launch ApiDemos. public void testNavigate() throws UiObjectNotFoundException { getUiDevice().pressHome(); UiObject allAppsButton = new UiObject( new UiSelector().description("Apps")); allAppsButton.clickAndWaitForNewWindow(); Press home button Click ‘All Apps’ button UiObject appsTab = new UiObject(new UiSelector().text("Apps")); appsTab.click(); Click ‘Apps’ tab
  • 140. Controlling UI Using uiautomator 4. Launch ApiDemos. public void testNavigate() throws UiObjectNotFoundException { getUiDevice().pressHome(); UiObject allAppsButton = new UiObject( new UiSelector().description("Apps")); allAppsButton.clickAndWaitForNewWindow(); Press home button Click ‘All Apps’ button UiObject appsTab = new UiObject(new UiSelector().text("Apps")); appsTab.click(); Click ‘Apps’ tab UiScrollable appList = new UiScrollable( new UiSelector().scrollable(true)); appList.setAsHorizontalList(); UiObject apiDemos = appList.getChildByText( new UiSelector().className(TextView.class.getName()), "API Demos"); apiDemos.clickAndWaitForNewWindow(); Click ‘ApiDemos’ icon
  • 141. Controlling UI Using uiautomator 4. Launch ApiDemos. public void testNavigate() throws UiObjectNotFoundException { getUiDevice().pressHome(); UiObject allAppsButton = new UiObject( new UiSelector().description("Apps")); allAppsButton.clickAndWaitForNewWindow(); Press home button Click ‘All Apps’ button UiObject appsTab = new UiObject(new UiSelector().text("Apps")); appsTab.click(); Click ‘Apps’ tab UiScrollable appList = new UiScrollable( new UiSelector().scrollable(true)); appList.setAsHorizontalList(); UiObject apiDemos = appList.getChildByText( new UiSelector().className(TextView.class.getName()), "API Demos"); apiDemos.clickAndWaitForNewWindow(); Click ‘ApiDemos’ icon Validate the launch of ‘ApiDemos’ UiObject apiDemosPackage = new UiObject( new UiSelector().packageName("com.example.android.apis")); assertTrue(“Should be on ApiDemos", apiDemosPackage.exists()); ...
  • 142. Controlling UI Using uiautomator 5. Find and act on a control.
  • 143. Controlling UI Using uiautomator 5. Find and act on a control. ... UiScrollable demoList = new UiScrollable( new UiSelector().className("android.widget.ListView"));
  • 144. Controlling UI Using uiautomator 5. Find and act on a control. ... UiScrollable demoList = new UiScrollable( new UiSelector().className("android.widget.ListView")); UiObject graphicsDemo = demoList.getChildByText( new UiSelector().className(TextView.class.getName()), “Graphics");
  • 145. Controlling UI Using uiautomator 5. Find and act on a control. ... UiScrollable demoList = new UiScrollable( new UiSelector().className("android.widget.ListView")); UiObject graphicsDemo = demoList.getChildByText( new UiSelector().className(TextView.class.getName()), “Graphics"); graphicsDemo.clickAndWaitForNewWindow();
  • 146. Controlling UI Using uiautomator 5. Find and act on a control. ... UiScrollable demoList = new UiScrollable( new UiSelector().className("android.widget.ListView")); UiObject graphicsDemo = demoList.getChildByText( new UiSelector().className(TextView.class.getName()), “Graphics"); graphicsDemo.clickAndWaitForNewWindow(); UiObject alphaBitmapDemo = new UiObject( new UiSelector().text(“AlphaBitmap")); assertTrue(“AlphaBitmap should be visible", alpahBitmapDemo.exists()); ...
  • 147. Controlling UI Using uiautomator 5. Find and act on a control. ... UiScrollable demoList = new UiScrollable( new UiSelector().className("android.widget.ListView")); UiObject graphicsDemo = demoList.getChildByText( new UiSelector().className(TextView.class.getName()), “Graphics"); graphicsDemo.clickAndWaitForNewWindow(); UiObject alphaBitmapDemo = new UiObject( new UiSelector().text(“AlphaBitmap")); assertTrue(“AlphaBitmap should be visible", alpahBitmapDemo.exists()); ... 6. Iterate step 5.
  • 148. Controlling UI Using uiautomator 7. Compile your test case into a JAR file. $ > $ $ android create uitest-project --name ApiDemosUiAutoTest --target 1 --path ApiDemosUiAutoTest cd ApiDemosUiAutoTest ant build
  • 149. Controlling UI Using uiautomator 7. Compile your test case into a JAR file. $ > $ $ android create uitest-project --name ApiDemosUiAutoTest --target 1 --path ApiDemosUiAutoTest cd ApiDemosUiAutoTest ant build 8. Install the jar on your test device. $ adb push bin/ApiDemosUiAutoTest.jar /data/local/tmp/
  • 150. Controlling UI Using uiautomator 7. Compile your test case into a JAR file. $ > $ $ android create uitest-project --name ApiDemosUiAutoTest --target 1 --path ApiDemosUiAutoTest cd ApiDemosUiAutoTest ant build 8. Install the jar on your test device. $ adb push bin/ApiDemosUiAutoTest.jar /data/local/tmp/ 9. Run the test and view the test result. $ adb shell uiautomator runtest ApiDemosUiAutoTest.jar > –c com.example.android.apis.uiauto.test.ApiDemosUiAutoTest
  • 151. Controlling UI Using uiautomator 7. Compile your test case into a JAR file. $ > $ $ android create uitest-project --name ApiDemosUiAutoTest --target 1 --path ApiDemosUiAutoTest cd ApiDemosUiAutoTest ant build 8. Install the jar on your test device. $ adb push bin/ApiDemosUiAutoTest.jar /data/local/tmp/ 9. Run the test and view the test result. $ adb shell uiautomator runtest ApiDemosUiAutoTest.jar > –c com.example.android.apis.uiauto.test.ApiDemosUiAutoTest $ ant test
  • 152. Controlling UI Using uiautomator 7. Compile your test case into a JAR file. $ > $ $ android create uitest-project --name ApiDemosUiAutoTest --target 1 --path ApiDemosUiAutoTest cd ApiDemosUiAutoTest ant build 8. Install the jar on your test device. $ adb push bin/ApiDemosUiAutoTest.jar /data/local/tmp/ 9. Run the test and view the test result. $ adb shell uiautomator runtest ApiDemosUiAutoTest.jar > –c com.example.android.apis.uiauto.test.ApiDemosUiAutoTest $ ant test
  • 153. Doing both Controlling and Tracking
  • 154. Doing both Controlling and Tracking 1. Modify org.kandroid.memtracer.MemoryInstrumentation. public class MemoryInstrumentation extends InstrumentationTestRunner { ... @Override public void onCreate(Bundle arguments) { ... start(); super.onCreate(arguments); } @Override public void onStart() { String mainActivityName = getMainActivityName(); if (mainActivityName != null && mainActivityName.length() > 0) { launchMainActivity(getTargetContext().getPackageName(), getMainActivityName()); } else { super.onStart(); } } ...
  • 155. Doing both Controlling and Tracking 1. Modify org.kandroid.memtracer.MemoryInstrumentation. public class MemoryInstrumentation extends InstrumentationTestRunner { ... @Override public void onCreate(Bundle arguments) { ... start(); super.onCreate(arguments); } @Override public void onStart() { String mainActivityName = getMainActivityName(); if (mainActivityName != null && mainActivityName.length() > 0) { launchMainActivity(getTargetContext().getPackageName(), getMainActivityName()); } else { super.onStart(); } } ...
  • 156. Doing both Controlling and Tracking 1. Modify org.kandroid.memtracer.MemoryInstrumentation. public class MemoryInstrumentation extends InstrumentationTestRunner { ... @Override public void onCreate(Bundle arguments) { ... start(); super.onCreate(arguments); } @Override public void onStart() { String mainActivityName = getMainActivityName(); if (mainActivityName != null && mainActivityName.length() > 0) { launchMainActivity(getTargetContext().getPackageName(), getMainActivityName()); } else { super.onStart(); } } ...
  • 157. Doing both Controlling and Tracking 1. Modify org.kandroid.memtracer.MemoryInstrumentation. public class MemoryInstrumentation extends InstrumentationTestRunner { ... @Override public void onCreate(Bundle arguments) { ... start(); super.onCreate(arguments); } @Override public void onStart() { String mainActivityName = getMainActivityName(); if (mainActivityName != null && mainActivityName.length() > 0) { launchMainActivity(getTargetContext().getPackageName(), getMainActivityName()); } else { super.onStart(); } } ...
  • 158. Doing both Controlling and Tracking 2. Add ‘kmemtracer-libs.jar’ in the ‘libs’ directory.
  • 159. Doing both Controlling and Tracking 2. Add ‘kmemtracer-libs.jar’ in the ‘libs’ directory. 3. Edit the <instrumentation> element in the manifest file.
  • 160. Doing both Controlling and Tracking 2. Add ‘kmemtracer-libs.jar’ in the ‘libs’ directory. 3. Edit the <instrumentation> element in the manifest file. 4. Build and Install the test package. $ cd ApiDemosInstrumentTest $ ant debug $ adb install –r bin/ApiDemosInstrumentTest-debug.apk
  • 161. Doing both Controlling and Tracking 2. Add ‘kmemtracer-libs.jar’ in the ‘libs’ directory. 3. Edit the <instrumentation> element in the manifest file. 4. Build and Install the test package. $ cd ApiDemosInstrumentTest $ ant debug $ adb install –r bin/ApiDemosInstrumentTest-debug.apk 5. Run the test and see the result. $ adb shell am instrument -w > com.example.android.apis.instrument.test/ > org.kandroid.memtracer.MemoryInstrumentation
  • 162. Doing both Controlling and Tracking 2. Add ‘kmemtracer-libs.jar’ in the ‘libs’ directory. 3. Edit the <instrumentation> element in the manifest file. 4. Build and Install the test package. $ cd ApiDemosInstrumentTest $ ant debug $ adb install –r bin/ApiDemosInstrumentTest-debug.apk 5. Run the test and see the result. $ adb shell am instrument -w > com.example.android.apis.instrument.test/ > org.kandroid.memtracer.MemoryInstrumentation 6. Pull the trace file ‘kmemtrace.csv’ from /sdcard/kmemtracer. $ adb pull /sdcard/kmemtracer/kmemtrace.csv .
  • 164. SUMMARY Why Automated Performance Testing? Measure to improve / Fragmentation
  • 165. SUMMARY Why Automated Performance Testing? Measure to improve / Fragmentation How to Automate Performance Testing? - Historical Analysis w/ kmemtracer Instrumentation for track
  • 166. SUMMARY Why Automated Performance Testing? Measure to improve / Fragmentation How to Automate Performance Testing? - Historical Analysis w/ kmemtracer Instrumentation for track - Measuring Performance API (Public / Internal) / cmds (Android / Linux)
  • 167. SUMMARY Why Automated Performance Testing? Measure to improve / Fragmentation How to Automate Performance Testing? - Historical Analysis w/ kmemtracer Instrumentation for track - Measuring Performance API (Public / Internal) / cmds (Android / Linux) - Controlling UI Instrumentation / Robotium / uiautomator
  • 168. Q&A