+5We have enabled a/b experiments and are testing what experiment performs best. We also have analytics around what SKUs users are seeing. We’ve noticed that our default variant is seen 70% of the time and variant B 30%. It’s hard to tell if this is a RevenueCat issue or whether our users are that skewed in terms of who is looking at the paywall.
Any ideas on why we would see such an uneven distribution?
Best answer by ryan
There should be some metrics in the Experiments dashboard on the “Number of users in variant” - does that appear to be a 50/50 split?
Are your analytics taking into account when you started running the experiment? All of the downloads from before the experiment started would still be seeing the default Offering which may be skewing your analytics in that direction.
Once you start an experiment, RevenueCat starts splitting the default Offering for new installs, it won’t change the Offering for existing installs.
+9There should be some metrics in the Experiments dashboard on the “Number of users in variant” - does that appear to be a 50/50 split?
Are your analytics taking into account when you started running the experiment? All of the downloads from before the experiment started would still be seeing the default Offering which may be skewing your analytics in that direction.
Once you start an experiment, RevenueCat starts splitting the default Offering for new installs, it won’t change the Offering for existing installs.
+5Thanks, that looks like it may be the issue. Most of our users are older and therefore on the default offering and therefore represents a larger portion of total users.
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.