Question

Data for the experiment is not available for almost 3 weeks

  • 7 February 2022
  • 4 replies
  • 77 views

Badge +4

Hi,

I started my experiment on January 20. Both variants have successfully converted users from 3-day trials. However, after almost 3 weeks (on February 7), I didn’t see any results for the experiment. Instead of it, I see the message “Data for this experiment will be available in 24 hours”. 

Questions:

  1. How do I know if my experiment is running successfully and I’m not wasting my time by looking at placeholders for several weeks?
  2. Why does RevenueCat say that the data will be available in 24 hours if this is never true? I’m running my third experiment and data was available in 1-3 weeks.
  3. What are the minimum criteria for data to be available? E.g., at least three converted users / purchases?
  4. Are there any plans to show how the LTV is calculated for each variant? 

Any clarifications on my questions would be greatly appreciated.

 


4 replies

Userlevel 3
Badge +8

Hi @AntonC,

For your specific case I noticed we’re seeing an error where the product duration for one of your products in the experiment is unknown. Judging by the product name, it looks like a non-subscription product (a one-time product) - we rely on the SDK to send product durations at the time of purchase, but in some cases where there is no duration, we estimate the duration based on having enough transaction data (20 transactions). I’m only seeing 11 transactions for this product so far. 

You can either wait for 20 transactions, or we can manually update it in our database for you (let me know if you want this and we can turn this into a Zendesk ticket). 

For anyone else wondering why their experiment data isn’t showing up, we have a guide here that may help you: 

 

Badge +4

​Hi @sundeep ,

Thanks for the clarification. I would much appreciate it if you could manually update it in the database.

Badge +4

Hi @AntonC,

For your specific case I noticed we’re seeing an error where the product duration for one of your products in the experiment is unknown. Judging by the product name, it looks like a non-subscription product (a one-time product) - we rely on the SDK to send product durations at the time of purchase, but in some cases where there is no duration, we estimate the duration based on having enough transaction data (20 transactions). I’m only seeing 11 transactions for this product so far. 

You can either wait for 20 transactions, or we can manually update it in our database for you (let me know if you want this and we can turn this into a Zendesk ticket). 

For anyone else wondering why their experiment data isn’t showing up, we have a guide here that may help you: 

 

Hi @sundeep 
Are there any updates on the issue?

Userlevel 3
Badge +8

@AntonC I’ve submitted a ticket with some follow-up questions - you should see it in your email (let me know if you can’t), we can follow-up there!

Reply