An app has to do a lot of things for you to love it—and do only two things wrong to make you hate it: perform sluggishly and consume a lot of power.
Facebook operations engineer Antoine Reversat revealed what Facebook does to prevent those two things and keep users from uninstalling its apps. He showed me the automated mobile testing lab, which has never before been disclosed to the public, that he designed and operates. The system automates testing of Facebook’s mobile apps, Messenger and Instagram for iOS and Android.
Almost 2,000 Android and iOS smartphones are housed in 60 racks at Facebook’s Prineville, Oregon, data center. The standard-sized, specially designed racks hold 32 phones each, interconnected to a server. Linux servers interconnect Android phones, and Mac Minis interconnect iPhones. The phones are remotely controlled and monitored during the testing using an automated system that, like much of what Facebook does, will be open sourced for other mobile testers to use and contribute improvements.
The system automatically installs apps for testing, monitors the apps, measures the execution of the apps and reports the test results to the Facebook developer, who can be at any of Facebook’s software development locations. The test phones are instrumented to measure the performance and power consumption of the apps. Called a regression, the test compares the app’s performance and power consumption to the baseline of the previous version. If the app runs slower or consumes more power than that baseline, the developer is asked to fix it.
The developer doesn’t take the initiative. Every twentieth iteration of the app, called a commit, is pulled from the software repository, installed on some of the phones and tested. While Facebook’s apps seem static and this scale of a test lab unnecessary, they are not. Apps are dynamic because of the mobile designers’ and developers’ fanatic zeal for testing new features, user interface subtleties and new user interactions within the apps.
Updates to the apps are frequently released to selected cadres of users for user experience testing. The users’ experience is measured with instrumentation built into the app, and the results are compared to the stable release to determine if the users like the changes and new features. If the tests prove a better user experience, the limited test version is released to Facebook’s user population. The mobile test lab ensures that the app’s performance and power consumption don’t drift in a sluggish or power-draining direction that could raise users’ ire.
Facebook's test racks
The racks that house the test phones look like any of the thousands of racks in Facebook’s 350,000-sq.-ft. lights-off data center, except they are fully enclosed. The enclosure prevents the Wi-Fi access points connecting the 32 phones in each rack from leaking and interfering with phones under test in the other racks.
Sign up for CIO Asia eNewsletters.