Share on facebook
Share on linkedin
Share on twitter
Share on facebook
Share on linkedin
Share on twitter

Measure Marketing Campaign Success with Control Groups

How effective are your marketing campaigns? If you know your campaigns are working, you can keep doing more of the same. Otherwise, you need to stop, rethink and try something different. Using control groups is a great way to test campaigns – but what is a control group, and how do they work?


What Is a Control Group?

In a marketing context, a control group is a subset of a selected audience segment that will be excluded from receiving a specified campaign message or promotion during a test period. A control group is what science calls the experiment placebo.

A control group should be picked at random to represent the entire segment. This enables like for like comparisons. It should also be relative to the number of people in the segment. For example, in a segment of 10,000, 5 percent or 500 people, would be sufficient.

As a general rule, smaller campaigns demand a higher percentage to develop a control group. So, if your target audiences were less than say 2,000 customers, your control group should be around 10 to 20 percent.

Another variable to consider when determining the size of your control group is expected response rate. If you are targeting dormant customers and are therefore expecting a low response, you need a larger control group to produce a statistically significant result.


Why Is a Control Group Important?

A control group allows you to measure the impact of your marketing campaign on a group of specific audiences. By comparing the responses of your control group with everybody else in the segment who was exposed to your campaign (the test group), you can determine its effectiveness.

Using a control group, it is possible to compare the measurable and meaningful KPIs that actually have an impact on your bottom line, such as profits and ROI. It also allows you to check whether your efforts are doing more harm than good – adding a third variable to traditional A/B testing.

Using a control group may seem counterintuitive. After all, by removing a percentage of your audience, you are setting yourself up for a loss in revenue. However, this short-term loss is outweighed by the opportunity it offers to make better, more confident strategic decisions.


Are Control Groups Needed for Every Marketing Campaign?

While control groups are useful for gaining marketing campaign insights, they are not necessary for every campaign.

One particular instance in which control groups are not necessary or effective is when targeting very small audiences of only a few hundred people. That is because the size of the control group in these cases would be too small to be statistically significant.

In addition, using control groups for all your marketing campaigns could potentially result in a substantial loss in revenue. It would be sufficient as long as you use control groups intermittently across campaigns to ensure that your insights are still valid. 

How Does a Control Group Work?

Implementing a control group in your marketing is a scientific but relatively simple process. Here is how it works step by step with an example:

1. Select a customer segment

Decide which group of customers you would like to target with a campaign. For example, you might decide to target 7,000 customers who haven’t purchased in three months by sending them a promotion offering a 20 percent discount.

2. Send campaign to your segment

Send your campaign to everyone in your chosen segment or test group (6,650 people), excluding the people in your control group (350 people). By excluding your control group, these customers can continue to act as they usually would, unaffected by the promotion.

3. Evaluate the results

Once your test period is up, compare the results of everyone in your test group to the results of those in your control group.

If 10 percent of your control group made a purchase but 25 percent of those in your test group who received the campaign message with the discount made a purchase, it would be a good indicator that your campaign worked. However, if your control group and your test group respond in a similar way, it is likely that your campaign was ineffective.

In each experiment, it is important to consider whether your results are statistically significant or not, meaning if the differences in your results are marginal. For example, if 25 percent of customers in your test group made a purchase, but 23 percent in your control group made a purchase, and then you should run another test to compare results.

4. Use findings to adjust your marketing strategy

Over time, you will start to see patterns in terms of what campaigns or offers drive the most revenue or deliver the most ROI – and which segments are most responsive. Using these insights, you can then tweak your future marketing strategy accordingly. 


Using control groups is a great way to measure the effectiveness of your marketing campaigns and make better strategic decisions. When marketing control groups are combined with customer analytics and machine learning platforms, they can be even more effective.

* Have a question? Or do you want to learn more about how to leverage machine learning for effective segmentation? Get in touch with our team today for an exclusive consultation. 



Let us know the marketing challenges that you’re facing, and how you want to improve your marketing strategy.


Data-Driven Marketers’ Guide to Data Management

Marketing has always been highly dependent on data. And in today’s rapidly moving world, the importance of managing vast quantities of diverse data from disparate sources is growing. “There are three main issues when it comes to data management for marketers,” explains Dr Min Sun, Chief AI Scientist at Appier. “The first is data quality – the maxim of ‘garbage in, garbage out’ is critical for marketers. Once data quality is assured, the second factor to consider is the usefulness of the data. What data is valuable and which data sets can be used together? Finally, companies need to ensure they have robust governance in place. These cover the legal and corporate obligations including jurisdictional frameworks, such as General Data Protection Regulation (GDPR) in Europe, the Personal Data Protection Act (PDPA) in Singapore and others.”   What Is ‘Good’ Data? There are several dimensions to data quality. Dr Sun says that issues such as consistent errors and inconsistent noise in data can significantly decrease the usefulness and value of even huge data sets. However, a more subtle issue can come even when the data is completely correct, but the assumptions underlying the selection criteria are biased or skewed. As a

How Consumer Insights Can Drive a Powerful Marketing Campaign

Marketing should be a two-way process. Instead of the traditional ‘top-down’ approach, it is vital to listen to your customers in order to meet their needs and provide experiences that tick all their boxes. By getting to know your customers, you will gain actionable consumer insights, and be much better placed to serve them more of what they want, and less of what they don’t.   What Are Consumer Insights? Consumer insights are understanding that a business gains about consumers and how they think and feel. More importantly, consumer insights shed light on why they think and feel this way. This gives companies an intimate knowledge of their customers, and helps them communicate with their audience much more effectively. If you know not only that a customer wants to buy a bright blue coat but also that they are doing so because they think spring colors suit them, you can recommend garments in other spring colorways. Consumer insights will help your business improve customer satisfaction and hence boost customer retention, all of which will increase your return on investment (ROI).   What Does It Take to Gain Meaningful Consumer Insights? Such insights do not come easy. Gaining actionable consumer insights

Do You Need a Data Scientist?

Author | Charles Ng, Vice President of Enterprise AI, Appier As brands evolve to become more sophisticated with their data, there is increasing pressure to add data scientists to their teams. But not every marketer may need to rush to hire. We are living in the age of data, and all savvy marketers know that leveraging data effectively is key to better understanding current customers and attracting new ones, driving towards long-term relationships and ultimately staying competitive. The amount of data available to marketers is vast and will only increase. At some point or another, marketing leaders are likely to have thought about whether they need a data scientist on the team to help them unify, manage and analyse data. Making this decision isn’t – or shouldn’t be – straightforward. There are several considerations for marketers before they seek out data science talent- something that, to begin with, is in short supply in many markets around the world. The decision to hire a data scientist is likely the result of an evolution. The first step might be to hire analysts, whose job is to make sense of the data, identify patterns and generate basic insights. Next, you may look at