Incorrect distribution of newcomers
Posted: Tue Jan 07, 2025 5:27 am
We had a case, during which we identified a strong bias towards loss. It turned out that the control group included people who had been in the database for the last 7 months, and the experimental group - from 7 months to 5 years.
Due to incorrect distribution, the results were incorrect. Clients who have been in the database for the last six months organically bring the company more money than those who have been in it for a long time.
Studying groups when calculating incremental macedonia whatsapp phone number revenue requires updating them correctly and in a timely manner. The segments being studied are dynamic and are constantly replenished with new clients. Adding newcomers to only one of the groups makes the results incorrect. And this is especially important when the experiment terms are long.
It is necessary to regularly replenish the database uniformly, maintaining the proportions of users who have long been entered into the system and recently arrived users in both groups.
One of our clients had a database of one and a half million people. Every month it received about 10,000 new clients, i.e. 1.5%. We divided the groups 95 to 5 (or 19 to 1), i.e. 1 million 425 thousand clients to 75 thousand. Only after some time did we discover a significant mathematical error that distorted the result.
Why were we wrong? It turned out that newcomers account for 20-30% of the total revenue in this company. Even one of them generates a thousand times more revenue than the average person in the database. We realized this in time and changed the proportions, which increased the homogeneity of the groups and helped collect accurate information about clients.
How to Calculate Incremental Revenue Correctly: Five Steps
Divide the customer base into two groups: experimental and control.
In the experimental group, start influencing the base using CRM mechanics, and work with the control group without promotion.
Divide the revenue in the experimental group by the number of people in it and do the same with the control group.
Subtract the second figure from the first to determine the difference in revenue between the two groups.
Multiply the difference by the number of people in the experimental group. The result will be the incremental revenue.
Due to incorrect distribution, the results were incorrect. Clients who have been in the database for the last six months organically bring the company more money than those who have been in it for a long time.
Studying groups when calculating incremental macedonia whatsapp phone number revenue requires updating them correctly and in a timely manner. The segments being studied are dynamic and are constantly replenished with new clients. Adding newcomers to only one of the groups makes the results incorrect. And this is especially important when the experiment terms are long.
It is necessary to regularly replenish the database uniformly, maintaining the proportions of users who have long been entered into the system and recently arrived users in both groups.
One of our clients had a database of one and a half million people. Every month it received about 10,000 new clients, i.e. 1.5%. We divided the groups 95 to 5 (or 19 to 1), i.e. 1 million 425 thousand clients to 75 thousand. Only after some time did we discover a significant mathematical error that distorted the result.
Why were we wrong? It turned out that newcomers account for 20-30% of the total revenue in this company. Even one of them generates a thousand times more revenue than the average person in the database. We realized this in time and changed the proportions, which increased the homogeneity of the groups and helped collect accurate information about clients.
How to Calculate Incremental Revenue Correctly: Five Steps
Divide the customer base into two groups: experimental and control.
In the experimental group, start influencing the base using CRM mechanics, and work with the control group without promotion.
Divide the revenue in the experimental group by the number of people in it and do the same with the control group.
Subtract the second figure from the first to determine the difference in revenue between the two groups.
Multiply the difference by the number of people in the experimental group. The result will be the incremental revenue.