Coverage checker test: Three mobile 
Spotting an opportunity to increase 5G broadband sales

Goal
Three wanted to offer 5G home broadband products to customers whose postcodes are returning a no coverage result on the 5G coverage checker because this is a missed sales opportunity. 

Problem
The coverage checker is flawed. Most people whose postcode returns a 'no coverage' result on the checker can actually get coverage - Three can tell this because the majority of users already have a 5G service with them. While it makes most sense to address the problem with the checker itself, this is a larger project. The interim measure was to test offering home broadband.

In addition, Three originally offered Mifi products to users showing a 'no coverage' results, so we had to measure the metrics accurately to make sure that offering home broadband was more profitable than pushing the Mifi alternative.

Hypothesis
We're confident the coverage checker is unreliable, so if we offer home broadband to users who have a postcode that's showing a 'no coverage' result, we'll increase 5G home broadband sales.

Method 
We ran an A/B test to customers with an equal split of traffic - 50% of users were sent to the control version and 50% to the test experience.

Experience A - control version
In the normal control view, users whose postcode returns a 'no coverage' result are shown two Mifi products as alternative solutions.

Experience B – test experience

With the test experience, the Mifi products are still shown, but the main body content has been tweaked to indicate that we think they can still get coverage and that it's worth trying because of no hassle returns.

Figma mobile and desktop view

An important metric for this test is also the return rate of the 5G Broadband, as we offer a free 30 day return for customers who buy it and find it doesn't work very well. We can't measure that in Analytics, but we can measure it in another database using the Target order IDs of those customers in the test group.

Results
We added a revenue view to easily evaluate how the uplift in 5G broadband is being offset by the drop in MiFi revenue - there's 52k gained in 5G revenue and only 20k lost in MiFi revenue.