Solved

Why do we have data discrepancies, even though Mixpanel uses RevenueCat events?

  • 26 July 2021
  • 2 replies
  • 37 views

Badge

We use Mixpanel as our Analytics tool, which has RevenueCat events fired for trial start and trial convert. However, our number of trial starts and trial converts, as well as the revenue in the latter event do not add up to match the numbers in the RevenueCat dashboard.

e.g.:

May: Trial starts in Mixpanel = 1,573

May: Trial starts in RevCat = 1,235

^ this is a 21% difference...

 

June: Trial converts in Mixpanel = 574

June: Trial converts in RevCat = 521

^ this is a 9% difference…

 

The event is from the same source, as we use the Mixpanel in-app purchases integration from RevenueCat https://docs.revenuecat.com/docs/mixpanel

 

I’ve used the data discrepancy article from RevenueCat, which doesn’t include the answer: 

 

Any ideas why these are different? This is crucial for our OKRs and revenue tracking

icon

Best answer by ryan 26 July 2021, 22:05

Two things I’ve seen before with the old Trial Start Rate chart and Trial Conversion chart that has caused confusion around data discrepancies are not taking the conversion windows into account when looking at other systems.

 

The old Trial Start Rate chart (below) is the percentage of trials started within 7 days of the customer being first seen in RevenueCat.

 

A better chart to compare against the number of new trials started in Mixpanel would be to look at the New Trials number in the Active Trials Movement chart (below).

 

 

Similarly, the old Trial Conversion chart in RevenueCat uses a 24hr conversion window. Meaning that a trial conversion is only shown in the chart if it goes from free to paid within 24hrs of the expiration date. In Mixpanel, a trial conversion event is dispatched even if it’s been days/weeks/month since the free trial expired.

A better chart to look at here would be the new Trial Conversion Funnel, or the New Actives number in the Active Subscriptions Movement chart (assuming all subscriptions start with a free trial).

 

Other things that usually cause smaller discrepancies are the differences in Mixpanel being an event-based analytics system and RevenueCat charts being snapshot based. Meaning things like refunds would remove an active subscription from the chart in RevenueCat, but events never get removed from Mixpanel. 

 

View original

2 replies

Userlevel 4
Badge +8

Two things I’ve seen before with the old Trial Start Rate chart and Trial Conversion chart that has caused confusion around data discrepancies are not taking the conversion windows into account when looking at other systems.

 

The old Trial Start Rate chart (below) is the percentage of trials started within 7 days of the customer being first seen in RevenueCat.

 

A better chart to compare against the number of new trials started in Mixpanel would be to look at the New Trials number in the Active Trials Movement chart (below).

 

 

Similarly, the old Trial Conversion chart in RevenueCat uses a 24hr conversion window. Meaning that a trial conversion is only shown in the chart if it goes from free to paid within 24hrs of the expiration date. In Mixpanel, a trial conversion event is dispatched even if it’s been days/weeks/month since the free trial expired.

A better chart to look at here would be the new Trial Conversion Funnel, or the New Actives number in the Active Subscriptions Movement chart (assuming all subscriptions start with a free trial).

 

Other things that usually cause smaller discrepancies are the differences in Mixpanel being an event-based analytics system and RevenueCat charts being snapshot based. Meaning things like refunds would remove an active subscription from the chart in RevenueCat, but events never get removed from Mixpanel. 

 

Badge

Super helpful! Thanks Ryan - numbers add up to within <10 users now

Reply