Skip to main content
Question

How experiments work with targeting


Forum|alt.badge.img+4

I have a question about how experiments work when targeting is configured.

Suppose I have a simple app which lets you track your hikes or bike rides. And I have two offerings: OfferingBike and OfferingHike which are shown to users depending on a custom attribute called preferredActivityType. This is just to make things easier.

One of these, say OfferingBike is set as default in console, although I never use this directly as I always set my custom attribute. So, in my case, the “default” offering is irrelevant.

Now I want to do an experiment. I want to try a new offering for bike riders. So I create a new one called OfferingBikeTest and adjust metadata. I say I want to test it against OfferingBike, which is set as default in my console.

I wonder how will the A/B test work in this case?

  1. Which users will be included? Will it be all users, or will I only get results from users who purchased a package belonging to either OfferingBike or OfferingBikeTest, ignoring those who make purchases through OfferingHike?
  2. Will such an experiment work correctly with targeting? What I mean is that if I have a rule called “show OfferingBike to users whose activity type is bike”, will those users be split into cohorts seeing OfferingBike or OfferingBikeTest?

Thank you.

This post has been closed for comments

2 replies

joan-cardona
RevenueCat Staff
Forum|alt.badge.img+6
  • RevenueCat Staff
  • 187 replies
  • July 16, 2024

Hi @andriyG,

Thank you for your post, very interesting questions here!

I want to start noting that experiments only work when you fetch the `current` offering, if you hardcode the offering id then it will always show that offering. So back to your example, rather than fetching OfferingBike or OfferingBikeTest, just get the current one when you want to get the bike one. This will ensure that the customers are evenly distributed through the experiment.

The experiment will be shown to all new users. We are working - but it’s not released yet - on targeting experiments. I can see how you might not get the full picture right now with experiments in your current set up but you could extrapolate the numbers if you know your distribution between bike and hike.

 

Best,


Forum|alt.badge.img+4
  • Author
  • New Member
  • 1 reply
  • July 20, 2024

@joan-cardona I assume the last paragraph implies that currently experiments do now work with targeting?

So, if I have a condition under which users would see OfferingBike and I create an experiment which tests OfferingBike against OfferingBikeTest - it will NOT be shown to users who qualify for OfferingBike based on targeting?

Right now when I test the app, upon first install, I do see both offerings from time to time.

However, on second launch, I always see the default offering and I never see the offering from experiment...


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings