Skip to main content
Question

How experiments work with targeting

  • 12 July 2024
  • 2 replies
  • 30 views

I have a question about how experiments work when targeting is configured.

Suppose I have a simple app which lets you track your hikes or bike rides. And I have two offerings: OfferingBike and OfferingHike which are shown to users depending on a custom attribute called preferredActivityType. This is just to make things easier.

One of these, say OfferingBike is set as default in console, although I never use this directly as I always set my custom attribute. So, in my case, the “default” offering is irrelevant.

Now I want to do an experiment. I want to try a new offering for bike riders. So I create a new one called OfferingBikeTest and adjust metadata. I say I want to test it against OfferingBike, which is set as default in my console.

I wonder how will the A/B test work in this case?

  1. Which users will be included? Will it be all users, or will I only get results from users who purchased a package belonging to either OfferingBike or OfferingBikeTest, ignoring those who make purchases through OfferingHike?
  2. Will such an experiment work correctly with targeting? What I mean is that if I have a rule called “show OfferingBike to users whose activity type is bike”, will those users be split into cohorts seeing OfferingBike or OfferingBikeTest?

Thank you.

This post has been closed for comments

2 replies

joan-cardona
Rank
Userlevel 2
Badge +5

Hi @andriyG,

Thank you for your post, very interesting questions here!

I want to start noting that experiments only work when you fetch the `current` offering, if you hardcode the offering id then it will always show that offering. So back to your example, rather than fetching OfferingBike or OfferingBikeTest, just get the current one when you want to get the bike one. This will ensure that the customers are evenly distributed through the experiment.

The experiment will be shown to all new users. We are working - but it’s not released yet - on targeting experiments. I can see how you might not get the full picture right now with experiments in your current set up but you could extrapolate the numbers if you know your distribution between bike and hike.

 

Best,

Joan.
A
Badge +4

@joan-cardona I assume the last paragraph implies that currently experiments do now work with targeting?

So, if I have a condition under which users would see OfferingBike and I create an experiment which tests OfferingBike against OfferingBikeTest - it will NOT be shown to users who qualify for OfferingBike based on targeting?

Right now when I test the app, upon first install, I do see both offerings from time to time.

However, on second launch, I always see the default offering and I never see the offering from experiment...

Terms & ConditionsCookie settings