Android App performance - Europe 2015

Editor's Notes

• #2: TK: Hi Everyone and Thank you for joining us. Today we’re going to be talking about App Performance, specifically we’ll be looking at real examples of mistakes that some of the top apps make, so that you can avoid them. We’ll also give you a simple testing plan to improve your own app, before the app store reviews get ahold of them.
• #3: TK …I think the next 40 minutes will fly by and help everyone improve their mobile apps. When we started looking at app performance 3 years ago, NO one was talking about it. Since then, we’ve worked with some of the largest development teams in the world. We’ve worked with the largest social networks, shopping apps, games, OEMs and even Platform providers to bring performance to the foreground. This year alone, we’ve tested more than a 1000 apps, and continue to find Performance issues that we think we can use to teach you how to improve your app.
• #5: TK When we talk about performance, we focus on how the network connections are impacting customers. And it may have a bigger impact than you think. Data connections account for 40% of the battery life when the screen is on, and 70% of the battery life with the screen off. When we first started testing apps, we focused our testing on actively using the app, but shortly afterwards, we found the biggest impacts were when the apps were in the background or even just idle. As we look at examples today, I want you to have a little background information on the RRC state machine. The RRC state machine is essentially the antenna on the device, and many developers aren’t aware, that every time the antenna is opened, it says open for an extra 15 seconds after the last data is pulled down…. In the example here, you can see that RRC state machine powers up (here in Red) and stays active after the first group of data comes down. When it stays open the antenna is at full power. Then as another group of packets comes down at 50 seconds, the antenna stays open again until it finally turns off at 64 seconds. While 24 seconds of antenna power isn’t extreme in this screen shot, We often see this behavior repeated for the entire time a user is in an app. 10, 15, 25 minutes or more of the app not giving the battery second to rest. I So Just keep that in mind as we look at the rest of the examples, We’ll be showing how the RRC state machine is affected with each example.
• #6: TK Some times we get push back …by focusing on the network, but honestly it’s the black box that can make or break your app. Just a 500 millisecond delay can increase user frustration by 26%, Decrease user engagement by 8%... And even a minor 100 millisecond delay can reduce Amazon’s revenue by 1% …. And our best stat of all is that High latency can lead to 4% of mobile users throwing their phones…. Admit it.. We’ve all been there.  Just think of your raise, when you walk into your bosses office and show them that you can reduce customer frustration, and increase engagement and revenue. It’s our job to help developers get there everyday…. Well, we can’t help with the raise, but the rest we can.
• #7: OK OK So your boss, doesn’t think “Performance” is something you need to focus on…… Well chances they do want you to focus on User Retention Rate, App store rating, User Acquisition Cost and things like Average Revenue per user. The more we research performance, the more we discover how performance influences all of these important application stats.
• #8: DS
• #9: DS
• #10: DS Repeatedly we hear, “But not my developers… They‘re the best…. That issue is so minor it won’t affect my customers….” 98% of all the apps we test have some sort of optimization What appear to be minor issues are responsible for killing user batteries and slowing down applications to a point where revenue is being lost…. “I can’t even test my own app without it being plugged into an outlet because it drains so much battery”
• #11: DS: Repeatedly we hear, “But not my developers… They‘re the best…. That issue is so minor it won’t affect my customers….” 98% of all the apps we test have some sort of optimization What appear to be minor issues are responsible for killing user batteries and slowing down applications to a point where revenue is being lost…. “There was a 4kb file that came down each time the app was launched and I didn’t think it was a big issue, until at launch we had so many downloads and new users that it took down our servers. Then we realized how much revenue we were losing every minute our servers were down.”
• #12: TK Those of us that come from a testing background, drive home “TEST TEST TEST. Test early test often” Sometimes I feel like I get a little preachy. But developers are getting hit hard by application reviews, and most users will only give you that one shot. We’ve got to put our best foot forward. Users are getting smarter too. While crash screens are obvious, users expect their apps to load in less than 4 seconds… Actually that’s the stat from a few years ago, it’s moving down toward 2 seconds. When I heard that, I immediately pulled out my phone and started launching apps to see how long they take to load. 4 seconds goes by quicker than you think and 2 seconds well… it’s tough. Try it tonight when you’re in your hotel, pull out your phone and launch an app that you would expect to be at the top of it’s game. Preferably one that you haven’t used in a while. And see how long it takes…. Put your phones away, I didn’t say do it now.
• #13: TK
• #14: TK: We all know Doug loves his farm animals. He actually does have a few goats at home, and we like to incorporate goats into all our presentations. We’ve had a little fun with the examples we’re showing today, and put goats in as often as possible. Find my Goat is an application that brings together Goats and riders. Riders request a goat and the goats mobile device lets the goat know where he needs to go. Find my Goat has seen tremendous success, but one day there was trouble in Goatville, when there was a sudden drop in sales reported. Image: https://www.flickr.com/photos/68919440@N05/6269603592/in/photolist-nuiq6T-bTKEXz-beHLNH-cWRPWA-7Ztgj6-bGT5rF-joN57z-irVKZ4-9y3tkW-8FDAqc-oKAvgN-buRoNN-6XLEt-ddRZJ-aEAwr-d2gnrh-gUbVCV-hqWXPa-b3FmqZ-hr81FU-92UxSM-aX9rHF-2VSy9v-negLWP-apVG8d-dNs5gT-dAog6U-dqmHYT-dH9Xfd-ei6orG-ay2mHE-bc6Ryg-fGnRGf-aRHExn-7JUXEc-9iy5ko-8X6JYv-bPzDS4-mS1Rbv-b3Fxuv-eXkN8A-b3FrRk-b3FsHT-aUusft-9CHUJ3-f47gFM-fZt3Sk-d9B6ps-f47fDD-6diGxW/ Usability starts with the design and architecture OS ie: Cross platform inter-relations (Drive app vs Customer app) Small issues become magnified No sandbox Tiny issues become exponential when multiplied by an entire platform Growth of product matrix needed early
• #15: TK: During a network test of Find my Goat, we found that the app was connecting to the network every 5 seconds……. So we know this is terrible for the battery, we can see that the RRC state machine is never able to rest, but on top of being hard on battery life. We discovered the app wasn’t reusing the same connection but creating a new connection each time…This turned out to be 18 times more connections than the architecture team expected and IP/port collisions were causing the loss in sales. Again, this is something that normal UAT testing wouldn’t show you. If you have 15 testers in a room, they may not have any issues, but when you are able to take an image like this to your architects and say “Are our servers ready to handle this for our 100,000 or more users?” it paints a different picture. You start to be able to see how these small connections start scaling. The other thing that I want to point out is the Throughput…. Notice how the throughput are a series of small blips. We often see this with streaming apps. We highly recommend grouping data, send out a larger batch of data, let the connection close and then reopen it. Often times we can cut loading time and reduce battery drain by just grouping data. Usability starts with the design and architecture OS ie: Cross platform inter-relations (Drive app vs Customer app) Small issues become magnified No sandbox Tiny issues become exponential when multiplied by an entire platform Growth of product matrix needed early
• #16: DS
• #17: DS: Netflix (Caching and Duplicates) (Can use example from Doug’s deck) Speed of app at startup Labs study where 17% of all content is dups How much of does that affect your servers if you remove 20% of all content served? Types of Caching Etags vs cache control Etags up to 3 seconds latency for connection
• #18: DS: Netflix (Caching and Duplicates) (Can use example from Doug’s deck) Speed of app at startup Labs study where 17% of all content is dups How much of does that affect your servers if you remove 20% of all content served? Types of Caching Etags vs cache control Etags up to 3 seconds latency for connection
• #19: TKI’ll be the first to admit that I love a good deal. I’m sure I’m the perfect demographic for these apps. We’re all bored with our standard date night,…sushi rolling lessons (Sweet), Painting and wine (I’m down for wine) , Goat racing….. Um, maybe…. http://pixabay.com/en/woman-smartphone-chatting-girl-410320/
• #20: TK We lovingly call thumbnails…. Dumbnails in our group. Downloading the same image twice, just doesn’t make since. Download it once and use it multiple ways. You can see these cute little Racing goats here, but when you click on the offer, you are faced with a blank screen. Meanwhile the app has requested a new image to be downloaded…. It’s the same image… 4 times larger…. Remember just 100 milliseconds decreases amazon’s revenue by 1% if the user is faced with a blank page for several seconds, chances are the user is going to not wait for that image to load. Reusing that initial image from cache would be displayed almost instantaneously. What we aren’t showing here is that when the user clicks on the goat racing deal, They encounter a blank page while that second image downloads… Image: http://www.buzzfeed.com/ailbhemalone/6-adorable-photos-of-the-oxford-cambridge-goat-race
• #21: TK Not all apps, are a companies first source of revenue. An executive of this company, said that this app was designed to help bring users into the store after browsing in the app. Great, so ease of browsing should be the top priority in this app. Image: http://pixabay.com/en/woman-shopping-happy-bags-dresses-169286/
• #22: TKWell… how many users are excited to see a whole screen of spinning circles. Each of these Dumbnails, I mean Thumbnails are 250 KB * 9 = 1.75 MB Progress indicators actually hurt the perception of how long it’s taking to load the images. At the very least remove the spinning circles, but there is so much more we can do here. Reducing the image size, and caching those images and again, just reuse them for the detail page. One level of loading screens is bad but two is worse.
• #23: DS
• #24: DS:Drive mode - GPS Battery impact Utilize existing API’s to identify user movements
• #25: DS: http://commons.wikimedia.org/wiki/File:Baby_goats_jan_2007_crop.jpg http://commons.wikimedia.org/wiki/File:Goat_in_a_car.jpg http://commons.wikimedia.org/wiki/File:2009_goat_and_bicycle_in_India_6512003001.jpg
• #26: TK Image: https://www.flickr.com/photos/perspective/33330283/ Usability starts with the design and architecture OS ie: Cross platform inter-relations (Drive app vs Customer app) Small issues become magnified No sandbox Tiny issues become exponential when multiplied by an entire platform Growth of product matrix needed early
• #27: TK I’m a football fanatic, my boys play, we have season tickets to our college games, and we love to be in Century Link field when our World champion Seattle Seahawks take the field. But when I’m in the stadiums, I cringe at the thought that every single phone for each of the 70,000 people in the stadium could have multiple apps that are all connecting absently to the network. This actually happens to the analytics on this app. The Developers of this app had no idea that this was happening all the time. When we test, we normally ask our testers to do a 20 minute idle test, but sometimes when we see this type of behavior, we ask to let the testers to let the test run overnight. We’ve even done 24 hour tests, again to show that this is a small issue that magnifies when you expand these connections over a 24 hour period for all your users.
• #28: TK App is IDLE. Ads downloaded every 30s. Files are not compressed (extra data, more time) Connections are not closed (blue bars) battery drain
• #29: TK
• #30: http://www.glendaleinquiry.ca/2014/09/what-were-we-thinking.html TK
• #31: TK
• #32: DS
• #33: DS
• #34: Text Compression Applications use text files in ways that the user wouldn’t expect and/or can’t see. Opportunities to gzip text files will be identified during a normal Active trace. Duplicate Content Testers will need to navigate back to some screens that they navigated through earlier in the trace to test caching and duplicate content. Closing Connections Opportunities to Close connections will be identified during a normal Active trace. Bluetooth If the application uses Bluetooth, it’s good to use the feature that launches Bluetooth early in the trace, as it will give an opportunity to see how long it stays on during the rest of the trace. GPS If the application uses GPS, it’s good to use the feature that launches GPS early in the trace, as it will give an opportunity to see how long it stays on during the rest of the trace.
• #38: DS