A/B testing for your app: useful tips

You launched an app. The concept is fantastic, you've done extensive audience research and maybe you've even thought about marketing your app. In short: everything is correct. Does your app still get bad reviews? Do the downloads fail or are the in-app purchases disappointing? There is a good chance that this is not due to the app itself. How can you then ensure that the (mobile) experience of your app improves and that you achieve the result you want? Correct. App A/B testing. We explain.

More conversion, more downloads and more in-app purchases. In short: more success for your app. Sound like music in the orders? Research has shown that app A/B testing accounts for a 124% increase in in-app purchases. Impressive! Not every app has in-app purchases as a conversion goal, of course. Maybe you have an app that offers study material to students such as the Examenoverzicht app, or an app that allows you (free) to ask all your business questions to experts such as with the LiveAdvize web tool. These apps have no in-app purchases, but do have other purposes. Think of the number of downloads and active users. Every app has a reason to get started with A/B testing.

With app A/B testing you test two versions of a component in the app. A/B testing is not always applied within app development. For online marketers, however, this is a piece of cake. Should the contact button be red or blue for more clicks? Which position works best for the contact form and should the titles be capitalized or not? Often these are small elements on a page or in an app, but they make for many differences. The same applies to apps: how do you ensure that users fill in the contact form more often, press that one button that you find important more often, play a game more often or place more orders? Especially with the regular updates of the Google or iOS operating systems, it is important to regularly perform app A/B testing. This way you know whether the update affects your conversion, or whether it is something else. You test your optimization ideas in an easy way without time-consuming (and expensive) technical requirements. With app A/B testing you analyze the effectiveness of your adjustments and do not base this on intuition or best practices, but on actual data.

You can test a number of components within your app through A/B testing. The largest 3 are app features and app content, push messages and in-app messages. We treat them:

App Features & Content

This category includes all content and functionality of your app. Most parts can be tested here. Does the app have a feature to allow users to view and share content? Then it is useful to see where the share and like button performs best. Do you have an app that serves as a service channel such as the Nexxt app we developed? Then it is relevant to see where the telephone numbers should be presented (and how!).

Push notifications 

With push notifications you can show users who are not in your app certain information, for example to direct them back to your app. Sending effective push messages is an art in itself. They are often perceived as a nuisance. It is therefore important that you send the right content with the right timing and the right destination. What is ideally suited for testing these push messages? App A/B Testing!

In-app messages

In-app messages are a perfect way to interact with your user. This works the same as the push messages, only the purpose is often different and the name says it all: it's in the app. In-app messages include sending offers, helping you navigate through the app or collecting valuable feedback. A small difference such as the chosen font can already yield a substantial difference.

Now that you know what app A/B testing is exactly, you will most likely want to apply it. There are dozens of tools you can use. We name a few that we as app developers regularly use during or after the app development process:

1. Firebase

Firebase is free for the hobbyists. You pay 25 dollars a month for the more advanced functions. You can register for free and use A/B testing. This tool is our favorite. With a clear interface / dashboard you can test all kinds of elements such as the previously discussed buttons, UI features or push messages. What makes this tool extra nice is that it is linked to analytics and Indexation, which means that you can directly monitor the effects of your tests. For example, it measures visitors, returning visitors, actions and engagement. Firebase supports both iOS and Android.

  2. Apptimize 

This tool is also available for both platforms: iOS & Android. The tool is similar to Firebase. You can select audiences and then target them. For example, by mobile device or location. This can be useful if you notice that a type of user converts a lot per tablet, for example. The huge popup (push message) that you see when you click on the link means that they also have something to optimize themselves.

Are you going to work with app A/B testing? Then the tips below can help you set up a successful A/B testing campaign:

• Clear goals form (SMART)

You may remember it from the past: SMART (Specific, Measurable, Acceptable, Realistic and Time-bound). Objectives are often formulated vaguely, which makes them difficult to measure. Make sure you know what you are measuring and that you can make it concrete.

• Keep it simple

A pitfall for many app A/B testing campaigns: making it too complex. When you test several adjustments at the same time, you do not know which result exactly has which part and you often achieve less return. Test small adjustments and iterate and optimize them.

• Optimize and repeat

Which brings us to the next tip: don't stop after you've tested something with an A/B test. Do you know that a blue button works better than a red one? Nice! Now test which text works better next to the color.

• Test per target group per device

A person aged 65+ can show completely different results than a teenager (logical). Yet this is often forgotten and this is shaved over 1 comb. Therefore always test per different target group. This way you know exactly what works for which target group, and you can respond effectively to this. It is important that you also take the device with you. A tablet user often uses an app very differently than one on a mobile device.

• To measure = to know

A term that has actually been used far too often, but that does not make it any less true. Do not blindly follow others or assumptions you have yourself. As likely as it sometimes seems, sometimes nothing could be further from the truth. A blue button may work better in the contact form, but in your push message a different color / text / tone, etc. can provide better results.

Due to the millions of apps in stores, it is important that your app stands out and converts. App A/B testing is therefore becoming increasingly important. This can already be done during the app development process, with interim peer reviews and user tests, but also after the release of an app. Actually, A/B testing is a continuous process, which can be further optimized. With a changing media landscape, target group that is constantly changing and operating systems that are updating, conducting targeted A/B tests has become indispensable. With the above information, you can apply your app A/B testing principles. Are you not yet achieving the desired result or would you rather have it done by an experienced team of developers and testers? Then take contact As a full-service app developer, we have extensive experience in conducting app A/B tests and we are happy to advise you.