Measure Marketing Campaign Success with Control Groups
How effective are your marketing campaigns? If you know your campaigns are working, you can keep doing more of the same. Otherwise, you need to stop, rethink and try something different. Using control groups is a great way to test campaigns – but what is a control group, and how do they work?
What Is a Control Group?
In a marketing context, a control group is a subset of a selected audience segment that will be excluded from receiving a specified campaign message or promotion during a test period. A control group is what science calls the experiment placebo.
A control group should be picked at random to represent the entire segment. This enables like for like comparisons. It should also be relative to the number of people in the segment. For example, in a segment of 10,000, 5 percent or 500 people, would be sufficient.
As a general rule, smaller campaigns demand a higher percentage to develop a control group. So, if your target audiences were less than say 2,000 customers, your control group should be around 10 to 20 percent.
Another variable to consider when determining the size of your control group is expected response rate. If you are targeting dormant customers and are therefore expecting a low response, you need a larger control group to produce a statistically significant result.
Why Is a Control Group Important?
A control group allows you to measure the impact of your marketing campaign on a group of specific audiences. By comparing the responses of your control group with everybody else in the segment who was exposed to your campaign (the test group), you can determine its effectiveness.
Using a control group, it is possible to compare the measurable and meaningful KPIs that actually have an impact on your bottom line, such as profits and ROI. It also allows you to check whether your efforts are doing more harm than good – adding a third variable to traditional A/B testing.
Using a control group may seem counterintuitive. After all, by removing a percentage of your audience, you are setting yourself up for a loss in revenue. However, this short-term loss is outweighed by the opportunity it offers to make better, more confident strategic decisions.
Are Control Groups Needed for Every Marketing Campaign?
While control groups are useful for gaining marketing campaign insights, they are not necessary for every campaign.
One particular instance in which control groups are not necessary or effective is when targeting very small audiences of only a few hundred people. That is because the size of the control group in these cases would be too small to be statistically significant.
In addition, using control groups for all your marketing campaigns could potentially result in a substantial loss in revenue. It would be sufficient as long as you use control groups intermittently across campaigns to ensure that your insights are still valid.
How Does a Control Group Work?
Implementing a control group in your marketing is a scientific but relatively simple process. Here is how it works step by step with an example:
1. Select a customer segment
Decide which group of customers you would like to target with a campaign. For example, you might decide to target 7,000 customers who haven’t purchased in three months by sending them a promotion offering a 20 percent discount.
2. Send campaign to your segment
Send your campaign to everyone in your chosen segment or test group (6,650 people), excluding the people in your control group (350 people). By excluding your control group, these customers can continue to act as they usually would, unaffected by the promotion.
3. Evaluate the results
Once your test period is up, compare the results of everyone in your test group to the results of those in your control group.
If 10 percent of your control group made a purchase but 25 percent of those in your test group who received the campaign message with the discount made a purchase, it would be a good indicator that your campaign worked. However, if your control group and your test group respond in a similar way, it is likely that your campaign was ineffective.
In each experiment, it is important to consider whether your results are statistically significant or not, meaning if the differences in your results are marginal. For example, if 25 percent of customers in your test group made a purchase, but 23 percent in your control group made a purchase, and then you should run another test to compare results.
4. Use findings to adjust your marketing strategy
Over time, you will start to see patterns in terms of what campaigns or offers drive the most revenue or deliver the most ROI – and which segments are most responsive. Using these insights, you can then tweak your future marketing strategy accordingly.
Using control groups is a great way to measure the effectiveness of your marketing campaigns and make better strategic decisions. When marketing control groups are combined with customer analytics and machine learning platforms, they can be even more effective.
* Have a question? Or do you want to learn more about how to leverage machine learning for effective segmentation? Get in touch with our team today for an exclusive consultation.
WE ARE HERE TO HELP
YOU MIGHT ALSO LIKE
Your travel or tourism enterprise has probably benefited from the recent upsurge within the sector overall. In 2017 alone, the industry contributed over US$8.27 trillion to the global market. So how can you use this demand to win not just short-term gains, but also long-term value? It’s not easy when — along with major growth — the travel and tourism sector has seen other changes too. The spread of online travel agencies, the rise of artificial intelligence (AI), and customers’ demands for unique destination experiences and seamless self-service all have a role to play in how you attract and retain customers. The Data Dilemma One challenge is dealing with the vast data generated in your operations, such as information about your customers’ travel destinations, preferences, budgets, and much more. Turning it into meaningful customer experiences can lead to higher conversions, stronger engagement and repeat business. But, even with all the data you have, there’s a whole element that may be missing from your efforts. Travelers are looking for an easy search process to find personalized and timely offers of just the right destination. For that, you need knowledge about their interactions elsewhere online, such as travel content consumed on
Recommendation engines are everywhere these days. In fact, some of the biggest brands we engage with every day are built around one, including Netflix, Amazon, Google, and Goodreads. Thirty-five percent of purchases on Amazon come from product recommendations. So, what is a recommendation engine, and how does it work? What Is a Recommendation Engine? A recommendation engine is a type of data filtering tool using machine learning algorithms to recommend the most relevant items to a particular user or customer. It operates on the principle of finding patterns in consumer behavior data, which can be collected implicitly or explicitly. Netflix uses a recommendation engine to present viewers with movie and show suggestions. Amazon, on the other hand, uses a recommendation engine to present customers with product recommendations. While each uses one for slightly different purposes, both have the same goal: to drive sales, boost engagement and retention, and deliver more personalized customer experiences. In the past, recommendations would come from a salesperson or friends and family. Today, we have passed this task in the hands, or minds, of algorithms. As a marketing tool, you could say that these machines are well-trained in the art of up-selling and cross-selling.
Many online businesses when encountering lower sales tend to focus on analyzing the causes of high shopping cart abandonment rates, instead of putting more effort into understanding online consumer behavior. You sure can leverage the quantifiable data and make changes to your e-commerce site, such as redesigning user interface and overhauling the content, but to gain a holistic view of your customers, you will need to understand their behavior throughout their buying journey. Only 2.58 percent of global e-commerce site visits were converted into purchases in the second quarter of 2019. This means, out of 34 online visitors, only one bought something. However, You can look at it as an opportunity to convert the remaining by identifying “What went wrong during the whole buying journey”. Following the four steps below, you can create an action plan on how to turn those indecisive online shoppers into paying customers. 1) Send Trigger Emails for Specific Customer Actions While exploring e-commerce stores, shoppers usually showcase various on-site activities and behaviors, such as subscribing to your newsletter, abandoning the cart, and browsing a specific category. As a marketer, you can leverage machine learning (ML) to analyze these behaviors and segment customers according to their