A/B testing mobile apps in 2025. Reality vs Expectations.

I was recently asked if I can provide some insights regarding A/B testing in mobile apps.

Thanks for the awesome questions to Samuray Kayo 🙌 (join my telegram community if you want to be part of the action too)

As this is something I’ve done extensively over the past 8 years, I decided to put my thoughts in writing.

Two important points before we begin:

1. This is not an extensive list of tools for A/B testing mobile apps (plenty of those online already). This list is focused on the stuff which I’ve used and I know works.

2. This is not a how-to guide. If there’s enough interest, I will put something together, but for the time being you’ll have to rely on other resources.

Two types of A/B testing solutions

Fully integrated

These are tools which allow for building (via some kind of visual editor), launching and analyzing A/B tests all in one. They usually include some kind of web editor, a segmentation and trigger setup and a dashboard for analysis.

Hybrid

These allow for everything you can do with the fully integrated type, except for the visual editor part. You can set audience segments, triggers and view the performance. But you need to build the visual part of the test within the app (yes, I mean - actually write code 🫠).

Of course - the fully integrated version is what we all want. But in reality they’re usually pretty hard to find, especially for smaller mobile apps.

Why no good A/B testing solutions exist, especially for smaller apps?

There’s a lack of accessible A/B testing tools for smaller app devs. The available solutions on the market are either too expensive or complex for a small app/team. From my POV the main reasons are:

• Smaller teams have other priorities and A/B testing is a nice to have, not a must

• Therefore A/B testing solutions for mobile apps are harder to find when you’re just starting out

• Most app sections (which are meaningful to be A/B tested) are difficult to abstract enough, so that someone would make a product out of that

Exceptions:

• Paywalls are easy to abstract and is one of the most important parts of an app to consider A/B testing. We’re already seeing some meaningful products on the market like Superwall and RevenueCat.

• Onboarding is also relatively simple to abstract, but I think we’ve yet to see a good product which would allow us to A/B test onboarding in drag-and-drop matter.

For everything else, your best bet is to look elsewhere, or build something custom.

What should you use?

Firebase A/B testing

Something as simple as Firebase A/B testing usually does the trick, especially if you’d like to include some kind of targeting or segmentation in the test. Their measurement could be a bit hard to understand at times, but it’s usually good enough to make informed decisions for most of the stuff you’d wanna test.

Build something yourself

If you want to do something even simpler than that - something as simple as a basic split test - you could set up something on your server. As most apps have some kind of backend logic, you could always serve a different value of a property at random. This is especially effective if you don’t want to rely on a separate SDK, or integrate the rather heavy firebase analytics SDK (= Google Analytics SDK) in your app.

Just make sure you have the ability to measure the results afterwards! Build a simple dashboard and measure the performance of the test. You can always extend it later.

If that’s not something you want to get into, you can always evaluate the results of the test in another way. For example: sell product A to group A and product B to group B (best use products you do not normally sell). Then analyse which one had more sales.

Remember, the results from the test and the learnings from them are far move valuable then a fancy platform.

Cheers and have fun!