P L E A S E  W A I T...

A/B Testing for Mobile Apps

According to research from Forrester, leading enterprises have consolidated online testing and personalization techniques such as behavioral targeting and recommendations into centers of excellence to achieve one common goal: to optimize digital experiences across the entire lifecycle of customer engagement.

Americans spend 90% of mobile internet time in apps, according to research conducted by eMarketer. Also, users spend two hours and 11 minutes per day using mobile apps, but just 26 minutes browsing the web on a mobile device. So it is clear that there is no competition between the app and web browser experience. But still, the number of enterprises investing in app A/B testing is low. The optimization for most organizations currently stops at app store A/B testing, and that itself proves to be a great ROI story.

For example, Rovio increased their conversions by 13% and got extra 2.5 million downloads for its game Angry Birds 2 by using app store A/B testing. App store A/B testing is where it all begins, but we are going to go a step further and talk about the inside app A/B testing in this article. We will use our experience working with Google Firebase for A/B testing (powered by Google Optimize) to shed some light on this topic.

    Right off the box, there is a definite ROI to be realized by implementing this tool:

  • Tracking of engagement, retention, and revenue is made possible, along with how user actions change per experiment (made possible through Google Analytics for Firebase).
  • The testing of changes in the UI, features, campaigns on a target group to measure changes to the top line before rolling it out to everyone who uses the app.
  • Utilization of statistical analysis like Bayesian to find out the best performing variations, which are also statistically significant.
  • Freedom for anyone in the team to run and analyze experiments, therefore saving up precious engineering time.

    Few of the case studies from popular apps say the same story. It is all about A/B testing when it comes to delivering the best customer experience on a mobile app:

  • 2X increase in user retention for Fabulous app by customizing the onboarding process via Firebase. Read more here
  • 5% increase in user sessions for Onefootball by increasing features testing before release. Read more here
  • Pomelo Games saw a 35% jump in revenue by implementing interstitial ads for all segments based on user behavior testing without losing players. Read more here

As you can see from the above examples, A/B testing for mobile apps does provide a definite ROI. However, we must remember the difference between platforms (Mobile app and website) and the added complexities:

1. Websites can typically handle a lot of client-side logic via JavaScript. It is, however, not the case with native mobile apps. A lot of client-side logic not only slows the user experience but is more likely to cause bugs.

2. When bugs do happen, they are much easier to fix on a website because shipping a new version of the site is easy. Users can see the version instantly — no need to wait for any third-party approvals. But, that is not the case with mobile apps. The version first needs to be approved by the app stores, and then, users need to update the app. A forced update is not a choice all the time. So, the timing is critical. However, testing does not have to wait. By leveraging Firebase Remote Config to show specific content to each defined audience, the need to deploy a new app update does not occur.

Drive better results by understanding customer data