NOTE: This is a legacy article describing the transition from the classic platform to ChurnIQ. Now, ChurnIQ is your default platform when you log into your Cleeng account. For more information, please refer to the Cleeng ChurnIQ section.
ChurnIQ is not only a new subscriber data platform at the analytics level. It is also a major change to the underlying data model and infrastructure on which your Cleeng analytics are built.
The new Amazon S3 data lake from which ChurnIQ’s data tables are generated provides much faster and more stable data processing. It also means that the data you see in ChurnIQ regenerates and refreshes more quickly, giving you a more up-to-date picture of your business. This has the biggest impact on analytics associated with subscriber and subscription statuses.
In addition to the new data infrastructure, here are some other reasons why your ChurnIQ and Classic metrics may differ:
- New classification rules, i.e. differences in how data is categorised
- Introduction of a subscriber centred view (greater focus on customers themselves)
- Slight changes in filtering logic
- Improved data aggregation processes - events like ‘churn’ and ‘new subscriber acquisition’ are now reflected in your analytics in a more logical way, and are more historically stable
Now we will describe four of the most common discrepancies you are likely to find in your analytics. These are:
- Differences In Active Subscriber And Churn Counts
- Differences in Cohort Analytics
- Differences In My Monthly Recurring Revenue
- Differences in Filtering Logic: Transactions Case Illustration
Use the links on the right to jump to a specific topic.
Differences In Active Subscriber And Churn Counts
Your active subscriber count is affected by multiple sub-counts in any 24 hour period. For example, your paying subscribers, your trial subscribers, your churned subscribers, or your winback subscribers .
There are some differences in the timing and classification of these events that will impact the exact counts you see on the ChurnIQ and Classic platforms.
For example ChurnIQ’s ‘Subscribe’ view uses your ‘subscribers’ as its unit of analysis, whereas the Classic Cleeng platform uses ‘subscriptions’ to create this count. A subscriber with multiple subscriptions would be counted multiple times in the old classification framework, whereas now they are recognised as one distinct customer account.
Ultimately a subscriber centred view is much more practical for managing your subscriber base. A customer with 2 subscriptions is still one customer after all, but this slightly changes the meaning of the metrics that you view. One example is that a cancelled subscription does not equal churn if that subscriber still has another active subscription. We will explain other factors that impact your churn count in the next section.
Example: Churned subscriber counts
A key factor is the precise classification of when events occur. Let’s take a churn event as our example. Within Cleeng Core, a subscription will not be terminated at precisely the moment when renewal should have occurred. As with any subscriber billing system, more than one attempt will be made to capture the payment for a subscription.
Why does this affect your subscriber count? Imagine a scenario where your customer’s subscription expires at 23:45 on January 31st. As the renewal scripts run at 23:30 (Jan 31) and 00:30 (Feb 1), the actual subscription termination may not take place until 00:33 on February 1st.
Can we say this subscriber churned in January or February? This can have a major impact on your active subscriber count from day to day, and from month to month. As users who in reality failed to renew on one day, could be counted as being active on the next.
ChurnIQ’s query logic accommodates such nuances in subscriber billing processes, and would in this case assign the churn event to the day that the subscription actually expired (Jan 31).
Importance of status classifications
In any given period of time, a customer may have multiple status changes, sometimes across multiple subscriptions. For example, in a two month timeframe, a customer may be in-trial, new, active, churned, and reactivated (won back).
Classification changes, while on the surface quite nuanced, can considerably change the numbers you see when compounded over tens (or hundreds) of thousands of customer events. What exactly is meant by ‘new’ (subscriber or subscription? inclusive or exclusive of winbacks?) might greatly change the subscriber acquisition reports on two different data platforms.
When refining our data models for ChurnIQ, we focused on adopting classification rules that:
- generated the most accurate up-to-the-moment picture of a subscriber base, and
- best aligned with the real cases we have experienced while working with over 250 broadcasters.
We therefore recommend that in the event of a discrepancy between our Classic and ChurnIQ analytics platforms, that ChurnIQ is adopted as your source of truth.
Differences in Cohort Analytics
Classification effects
Our updated classification rules will impact analytics such as subscriber cohorts. Take the following scenario:
- In January 2020, 3,000 new subscriptions start
- Of these 3,000 subscriptions, 2,000 are new customers & 1,000 are reactivated customers
Using a subscription view, your cohort size for January is 3,000. With a subscriber view, your cohort is 2,000.
Why? Because those 1,000 reactivated subscribers are part of earlier cohorts, having first signed up at some time in a previous year. Using a subscription view here will in practice assign individual customers to multiple cohorts. This makes it more difficult to track individual groups of customers over time, and indeed to monitor the effectiveness of the campaigns used to acquire those customers.
Within ChurnIQ cohort analytics, we use the preferred practice of assigning customers only to the cohort where they were initially acquired. This aligns with the mode of cohort analysis used in your Google Analytics dashboard or comparable CRM tools.
Customers churning and then starting another subscription will be reactivated within their original cohort, rather than being assigned to new cohorts as was the case in the subscription-centred Classic platform.
This provides a much clearer perspective of your real retention rates over time. In turn this provides you with a much better sense of what your typical subscriber lifecycle looks like, and how this is changing from month to month.
Differences In My Monthly Recurring Revenue
Slight differences in a data model’s structure will cause differences in the monthly recurring revenue metrics you will see on different platforms. This is also the case for the ChurnIQ and Classic Cleeng data platforms. There are 2 primary reasons you will see this difference:
- Greater sensitivity to subscriber status changes
- More frequent regeneration of your recurring revenue data model
Let’s look at these one at a time.
Sensitivity to Subscriber Status Changes
The biggest cause of differences in your Monthly Recurring Revenue (MRR) figures will be the greater sensitivity to subscriber status changes in ChurnIQ. As described in the explanation of changes to subscriber/churn counts above, ChurnIQ has a more advanced churn classification process which more accurately detects and more quickly reports that (1) a subscription has been terminated, and (2) that a subscription has been created.
As your MRR data builds on both transactional and subscription history data tables within ChurnIQ, these classification improvements will provide a more accurate picture of your MRR at any point in time.
More frequent regeneration of your recurring revenue data
It is important to remember that MRR is a simple forecast of expected revenue in the next month, assuming no changes in current contributions to MRR by customers. A significant change between the Classic Cleeng platform and ChurnIQ is that the timeframe for recognising these contributions has changed, and that a clearer revenue normalisation process is in place.
Let’s quickly review these.
Starting with the timeframe, the Classic Cleeng platform interpreted MRR in terms of a summarised view of the previous month’s transactional activity. This meant that it did not recognise events or status changes since a customer last successfully paid for a subscription. So if a customer transaction took place on the 5th of the month, it did not look at the intervening 25 days when calculating MRR at month end.
This approach is generally reliable, but is not as accurate as a methodology that daily re-checks the status of subscriptions that contribute to your MRR. This methodology is used in your new MRR calculation in ChurnIQ. The underlying data model for recognising your subscription revenues is daily re-aggregated from Cleeng’s Amazon S3 data lake.
This means that the MRR figures that you see no longer provide a generalised view based on the previous 30 days of activity, but instead show you an up-to-date picture of your expected recurring revenue as of the beginning of each day.
Differences in Filtering Logic: Transactions Case Illustration
You might also notice some differences in some simple metrics such as your transaction volume or transaction revenue. This is due to a slight change in how the filter logic of the Classic platform differs from ChurnIQ. We will use the transactions case to illustrate this change.
In the Classic platform, setting your filter on a transactions dashboard from the Feb 1st to Feb 7th would show you precisely the following data:
All transactions between 00:00:01 on February 1st and 23:59:59 on February 7th.
The date filter flexibility in ChurnIQ is much greater than in the Classic platform. In addition to a standard date range picker, you may also select date ranges as in the following examples
- in the past ‘x’ days/weeks/months/quarters (relative dates)
- is on the day eg.March 6th
- is before March 1st
- is after January 2nd
- is on or after February 1st
However, in order to main consistency between these different methods of date selection, the logic treats the beginning of the last day in selected dates as its cutoff when you use the ‘in the range’ date picking method. Not the end of that day. So to use the earlier example, setting your filter on a transactions dashboard from the Feb 1st to Feb 7th would show you precisely the following data:
All transactions between 00:00:00 on February 1st and 00:00:00 on February 7th.
To cross-check data between the two platforms, simply select one day later for the end of your date range in ChurnIQ.
To learn more about how ChurnIQ’s filter logic works, take a look here.