Skip to main content
Question

When is the right time to conclude an experiment?


Forum|alt.badge.img+3

I’ve read the article Experiments: Debugging and best Practices for A/B Testing that was probably written when Experiments were still in beta. The documentation mentions an “LTV Chart” with a gauge to determine when you can conclude an experiment.. However, I can’t find that my RevenueCat dashboard. 

My case: 
I have two subscription groups that both contain a monthly and yearly subscription. The trial length for the subscriptions between the groups are identical. The only difference is price.

I’ve been running the experiment for around two weeks now, and see positive results. However, it’s still unclear for me when I can finish the experiment. 

2 replies

Forum|alt.badge.img+8
  • RevenueCat Staff
  • 418 replies
  • January 5, 2023

The documentation mentions an “LTV Chart” with a gauge to determine when you can conclude an experiment.

 Sorry for the confusion, we need to update that document.

 

However, it’s still unclear for me when I can finish the experiment. 

 It is up to you when you want to conclude it, it depends per app but you should do so whenever you feel like you have the results you want. One thing to note is that when you stop an experiment, the results will continue to be updated for 400 days to capture any additional subscription events. Please see here for our tips on running experiments: https://www.revenuecat.com/docs/experiments-overview-v1#tips-for-using-experiments


Forum|alt.badge.img+6
  • Dedicated Member
  • 15 replies
  • February 22, 2024

“One thing to note is that when you stop an experiment, the results will continue to be updated for 28 days to capture any additional subscription events.”

Can you clarify? I’m unclear what data is added to the experiment after it is stopped. If I stop an experiment and then all the users in the experiment get their offering changed to the default offering, then what data can you continue to collect that won’t mess up the test (having changed the offering)???

 


Reply


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings