Share on facebook
Share on linkedin
Share on twitter
Share on facebook
Share on linkedin
Share on twitter

Data-Driven Marketers’ Guide to Data Management

Marketing has always been highly dependent on data. And in today’s rapidly moving world, the importance of managing vast quantities of diverse data from disparate sources is growing.

“There are three main issues when it comes to data management for marketers,” explains Dr Min Sun, Chief AI Scientist at Appier. “The first is data quality – the maxim of ‘garbage in, garbage out’ is critical for marketers. Once data quality is assured, the second factor to consider is the usefulness of the data. What data is valuable and which data sets can be used together? Finally, companies need to ensure they have robust governance in place. These cover the legal and corporate obligations including jurisdictional frameworks, such as General Data Protection Regulation (GDPR) in Europe, the Personal Data Protection Act (PDPA) in Singapore and others.”


What Is ‘Good’ Data?

There are several dimensions to data quality. Dr Sun says that issues such as consistent errors and inconsistent noise in data can significantly decrease the usefulness and value of even huge data sets.

However, a more subtle issue can come even when the data is completely correct, but the assumptions underlying the selection criteria are biased or skewed. As a result, the data collected can cause an AI model to produce poor results. While some bias in data selection can be useful, it is critical that the bias is recognized and understood.

“Once the bias is known, it is possible to un-bias the data to remove its effect on the machine learning and AI algorithms,” he adds. 

The frequency, recency and coverage of data also impact its quality. While both new and old data are useful, typically marketers will prefer newer data as it is more relevant. However, there are times when having both old and new data useful for trend analysis and understanding cause and effect. Having data covering a broad time horizon can be powerful for marketers, says Dr Sun. 


What Data Sources Can Marketers Use?

With so much data available to marketers, knowing which sources are most valuable and how they can be used is critical. In most cases, the data can be broadly split into two groups: customers’ online behavior data and offline behavior data. Both are equally valuable for understanding existing customers and making compelling offers to them.

“These days, marketers have access to a large volume of  online data, such as customer behavior from websites. For example, they can learn what products a customer buys, the completion of a transaction and then return visits,” explains Dr Sun. “However, there are also offline data sources. These can include visits to physical stores, phone calls made to customer centers and other information that is managed through a CRM.”

Typically, the data pool from online data is very large, but offline data sources like CRM and finance applications can be useful for helping marketers understand things such as how customers paid for purchases. This can help develop models to incentivize sales by knowing when to offer preferred incentives, such as discount coupons, free shipping or options for customers to use convenient payment methods like  ‘Buy Now, Pay Later’ services.

Businesses can also, through the use of APIs and other tools, reach audiences from other sources like social media, leveraging the knowledge extracted from their own data to interact with audiences on other platforms.

“We should combine the power of first-party data and the knowledge learned to reach various types of new customers through APIs of other platforms,” Dr Sun adds.


Pulling the Data Together 

Finding and collecting the right data is the start of the process. The next hurdle to overcome is integrating the data from different sources so it can be used effectively. That means finding common elements between the data from each source, so customer records are complete and there is no duplication. The ideal way to do that is to find a common identifier that can be used to link the data.

“The first step is to aggregate the data while maintaining data quality,” says Dr Sun. “Ideally, to match Data Point A from one source, to Data Point B from another source by finding a simple identifier.”

For example, your CRM data may identify someone by their name and cell phone number. And another online data source may also hold the cell phone number. In that case, you can reasonably assume that the cell phone number uniquely identifies the data as coming from the same person. Or you may need a combination of attributes to definitively determine that two records from different systems are for the same person.

In reality, things may be more complicated. People either accidentally or intentionally enter numbers incorrectly. Or they may use their full name in some systems but a pseudonym or abbreviated name in others. That can make the task of correctly aggregating the data quite complex.

Another factor in aggregating the data is timing. When two data records for the same person exist, it is important to know which is most recent. However, Dr. Sun points out that it is also important to understand that the most current record may not always be the one you want. Two data records may be indicative of a process where one record shows an effect while the other reflects the cause. In such cases, both records are valuable. 

It is also possible that the data you are bringing into your customer data platform is a subset which can result in bias depending on the selection criteria.

“Once you have the data, you can put it into the models and figure what data is useful and if there is missing data that would be valuable to be proactively collected for improving the models,” says Dr Sun.


Data Governance Is Crucial

A crucial element in data management is governance. This covers internal policies, processes and procedures which, in turn, are informed by jurisdictional and legal frameworks.

For example, the European Union’s GDPR requires that organizations holding data pertaining to its citizens follow a set of specific guidelines and rules And in Singapore, the PDPA has specific provisions regarding consent to collect and use information, limits on how data may be used and limits on retention. Many countries now have specific laws pertaining to the reporting of data loss. Australia has its Notifiable Data Breach notification scheme and New Zealand recently amended its data privacy laws.

In practice, this means organizations must be very clear to customers about what data is being collected, how it will be used and have a process in place for data deletion, as well as robust information security. This would be covered within a data governance policy or framework that clearly states this process and makes it clear who has responsibility for different elements of the data governance policy.

Data governance is about much more than compliance. Good data governance is about applying best practices that cover everything from data architecture to security and data retention, ensuring there are appropriate procedures in place to support the best possible data quality. From a governance perspective, this means taking a lifecycle view of data.

If data is the new oil, then it is incumbent on marketers to ensure they collect, refine and use data as a precious resource. Ensuring that marketing data is governed correctly with respect to jurisdictional obligations is also critical. There is a cost to collecting, storing, managing and using data. It is a tangible asset that can support marketers in their ability to understand customers and make them compelling offers that result in increased sales.


* Read our latest white paper ‘The Rise of CDPs and Their Critical Role in Digital Transformation’ to learn more about how you can leverage a customer data platform to turn data into actionable insights for improving customer experience. Got a question? Contact us today!


Let us know the marketing challenges that you’re facing, and how you want to improve your marketing strategy.


Appier Celebrates 5 Years in AI

Appier celebrates our 5th Anniversary this year.  We share some milestones of the company’s progress in our journey towards Enterprise AI in this infographic. Appier – then and now About Appier Appier is a technology company which aims to provide artificial intelligence (AI) platforms to help enterprises solve their most challenging business problems. Appier was established in 2012 by a passionate team of computer scientists and engineers with expertise in AI, data analysis and distributed systems. Appier serves around 1,000 global brands and agencies from offices in 14 markets across Asia, including Taipei, Singapore, Kuala Lumpur, Tokyo, Osaka, Sydney, Ho Chi Minh City, Manila, Hong Kong, Mumbai, New Delhi, Jakarta, Seoul, and Bangkok. For more information please visit

Making 1:1 Personalization Possible With AI

The way a company markets itself and its products or services can have a huge impact on its profits, brand strength and longevity. While a poorly planned and executed marketing campaign is unlikely to help a company greatly, using modern techniques such as personalization powered by artificial intelligence (AI) can push your campaigns to the next level.   Marketing Personalization Is A Necessity While many brands have already started to personalize their messages using marketing automation tools, online marketing allows for much more specific targeting, almost on an individual basis. Potential customers leave breadcrumbs of their interests across the internet, through social media, shopping sites and search engines. This data can help you craft a better marketing campaign, one with tangible benefits. Personalization at scale can lift revenue by 5 to 15 percent for companies in the retail, travel, entertainment, telecom and financial services sectors, according to McKinsey & Company. There is no surprise to see that personalization is becoming a major consideration as McKinsey highlighted that more than 90 percent of retailers believe this is a top priority. However, only 15 percent of these companies are actually doing a good job at it.   Challenges With Marketing Automation Tools

Is AI the Remedy for Brand Safety Woes?

When it comes to programmatic advertising today, companies are focusing on brand safety as much as they are on impressions, click-throughs and revenue generation, but it is virtually impossible to monitor brand safety due to the scale and speed that programmatic offers. However, with the increasing adoption of artificial intelligence (AI) in digital marketing, such technology will not only help marketers better target their ideal audience, it might just be the cure for protecting advertiser dollars.   Programmatic Could Compromise Your Brand Safety Brand safety came under the spotlight following a number of advertising mishaps in early 2017. Alexi Mostrous from The Times revealed that many household brands were unwittingly supporting terrorism on YouTube by placing their ads on hate and Islamic state videos. Later the same year, ads from some of the world’s biggest brands were seen to be running alongside videos that sexually exploited children. This led to widespread panic, with many brands pulling programmatic spend until they could be assured by publishers like Google that measures were being taken to filter out such content. In 2018, brand safety has broadened to cover any offensive, illegal or inappropriate content that appears next to a brand’s assets, thus threatening