Share on facebook
Share on linkedin
Share on twitter
Share on facebook
Share on linkedin
Share on twitter

How Emotional AI Can Benefit Businesses

Author | Min Sun, Chief AI Scientist, Appier

Emotion is one of the most distinguishable human qualities, one that sets us apart from machines. However, it is not out of the realm of possibilities for machines to read emotions and respond accordingly. Increasingly, machines are able to interpret human’s emotional states and adapt their behavior to give appropriate responses – something we call emotional AI, or artificial emotional intelligence (though in the computing field, it is known as affective computing).

Here we will explore what it is, how it works, and how it can benefit businesses.


Three Types of Emotional AI

Emotional AI is the next step in the evolution of artificial intelligence. By interpreting people’s emotions, AI can respond in a much more naturalistic manner, making the interaction much closer to typical human intercourse.  

There are three main types of emotional AI – natural language text analysis, voice analysis and facial expression analysis. The first two are already quite common, while the third probably attracts the most media attention. Other types of analysis also include mouse movement, eye-gaze, heart rate and electrocardiography, etc.

Natural language text analysis

It involves AI scanning written text like a review of a product or service, online articles or tweets, and then picking up on the sentiment of whether it is positive, negative or neutral.

Voice analysis 

This analyzes a user’s speech signals like their vocal pitch, intonation and tone as well as the words they use to determine their sentiment. For example, someone with a dry sense of humor might say the opposite of what they actually mean for comic effect, but using voice analysis you could pick up on the true meaning of what they are saying. This is especially useful in call centers – detect an angry tone of voice from a caller, and you can transfer them to a human operator rather than risk frustrating them further by making them deal with an automated system.

Facial expression analysis 

This is perhaps the most interesting one. Using a video camera to read someone’s facial expressions, AI can analyze their emotions, and from that you can infer their state of mind, their intentions, whether they are lying or being genuine, and so on. Some startups already use this in their job interviews to determine whether the interviewee is nervous, confident, or sincere about his answer, etc. It also has enormous potential for financial services companies, such as banks or fintech firms, when they are deciding whether to approve a loan for someone.


Analyzing Emotions for Retail: Online and In-Store

Emotional AI will be useful to all kinds of businesses. If your company needs to understand human emotion in order to make a decision, you will have a use for emotional AI, as it can help automate this analysis and hence these decisions. Nowhere is this truer than in the retail space.

Using cameras in stores, emotional AI can observe your customers’ facial expressions, how they walk, and other variables that will help determine their emotional state. For example, if someone is frowning a lot and walking at a very fast pace, you could deduce they are stressed and in a hurry. In which case, you could advise your salespeople not to approach them to tell them about your latest offers.

We can do the same thing online. Instead of seeing a shopper’s body language you can analyze their online behavior patterns. If they use their mouse cursor very aggressively, for instance, AI can infer they are stressed and/or in a hurry, and less open to offers. On the other hand, if they hover the cursor over the ‘Buy now’ button for a while, they might be indecisive, and so it could be a good time to send them a coupon for a discount or free shipping to help convert them into a paying customer.


Nailing an Emotion: Accuracy and Interpretation

So how accurate is emotional AI?

In the case of facial expression analysis, researchers have defined about 64 different facial expressions and micro expressions. AI can detect these with a pretty high degree of accuracy, and this will only improve as the technology develops – like other forms of AI, emotional AI improves as more data you feed it. So, accuracy isn’t the issue. 

Problems arise when you get onto the question of interpretation. Just because someone pulls a certain facial expression in response to a certain question, that doesn’t necessarily mean they are lying. It is a big leap to go from recognizing a facial movement usually associated with one kind of emotion or behavior, to assigning a certain motive to that expression. People’s emotions can be quite unique, and different people show emotions differently – there is a huge amount of variation in how people react to certain situations. You have to go person by person, and be wary of devising universal rules that you apply to everyone the same.

In demos, the facial expression recognition feature of emotional AI looks very impressive. That is because in those demos, researchers asked actors to pretend to be nervous, and AI picked up on that emotion. However, actors tend to over-emote – in the real world, people behave very differently. In real scenarios, I would say it is accurate 70 percent of the time, but use an actor and that rises to about 90 percent.


Coupons Yes, Job Denials No: Putting It to Good Use

You probably shouldn’t use a system with 70 percent accuracy to make “final” decisions like whether to hire someone or deny them a loan. That is because you are taking the final decision out of their hands, which is a lot of responsibility for a company to wield. However, pairing humans and AI to make an improved final decision can already be useful in certain scenarios.

For instance, emotional AI is very well suited to marketing decisions like whether to send someone a coupon. With this kind of decision, the final say is still in the customers’ hands, rather than the company’s. You as a marketer are just nudging the customer to complete the purchase.

Marketers should start preparing for emotional AI by focusing on data that reflects your customers’ emotions. For example, a customer call center can record all calls, and a website can store all user reviews for analysis. Use this data, and soon you will be able to leverage this technology to make more effective business decisions.

It is still relatively early days for deploying emotional AI in the real world, but it is a crucial step in the development of AI, and absolutely essential if we want to build an AI that interacts naturally with humans.


* This article was originally published on Campaign Asia


Let us know the marketing challenges that you’re facing, and how you want to improve your marketing strategy.


COVID Has “Accelerated Digital Transformation From Years to Months”

12 brand leaders discuss the challenges of digital transformation at a virtual roundtable hosted by Campaign and Appier.   How should marketers invest in digital solutions that allow them to engage with their customers both in the short- and long-term? That was one question raised at Digital Transformation in Marketing: Boosting customer engagement with data and AI, a roundtable that looked at the various digital transformation challenges brands are facing in 2020. Hosted by Campaign in partnership with Appier, the session brought together marketing leaders from 13 brands: Citi, Aon, Fuji Xerox, Fairprice Group, Pizza Hut, Yum! Restaurants International, Love, Bonito, Naiise, The Wall Street Journal, Singapore Sports Hub, PropertyGuru Group, MoneySmart and Zespri International. Michelle Wong, SVP, Enterprise Solutions at Appier, kicked off the roundtable with a short presentation about the challenges facing marketers today. One of the major challenges in 2020 is the consumers shift to digital. This is accompanied by a willingness to try new apps and channels, increasing the number of potential touch points for marketers. According to findings by Appier, consumers in SEA are, compared to their North Asian counterparts, more likely to shop outside their home countries, with a staggering 85% willing to try

Ask the Experts: Rani Saad on the State of Digital Transformation

During a recent interview, Rani Saad, Executive and Learning coach for Digital Transformation and Innovation at INSEAD, shared with us his view on digital transformation and how brands in the travel and luxury sectors can leverage AI-driven tools to tackle their challenges, especially during uncertain times like now.    Tell us a bit about your background and how you help businesses understand digital transformation and what true innovation is.  After I finished business school, I started my professional life in aviation and was part of the Air France acquisition of KLM. This was the beginning of the ‘digital life’ of these organizations, and I ended up in charge of e-services and figuring out how to execute digitally, which back then was essentially providing sales and services online. From there I supported other stakeholders in the travel value chain, including Changi Airport in Singapore, looking at what the future of air travel was going to look like. I’ve also been involved in telecoms, education, startups, government and so on, but what all these experiences have in common is that I’ve always been involved in helping organizations build not just a digital strategy, but to actually execute the strategy so that they

Omnichannel Retailing: What, Why and How

Today’s consumers want to interact with retail brands through multiple channels, be they online, mobile or in-store. Not only that, they also want an effortless shopping experience across all touchpoints. An omnichannel retailing strategy can help retailers meet this need, increasing customer engagement and, in turn, sales.   What Is Omnichannel Retailing? Omnichannel retailing is a methodology that allows you to be truly customer-centric, and ensures all your retail channels to work in harmony with each other. It breaks down the barriers between online and physical sales, creating a seamless shopping experience for the customer, regardless of where they are, what device they are using, and what channel they are accessing content from. With omnichannel retailing, inventories and supply chains are also managed as one single entity – like one central repository from which all retail channels can draw. That gives you as a retailer much more flexibility in terms of how you fulfill customer orders.   How Is It Different From Multichannel Retailing? While there are some similarities between the two, in practice they operate very differently. With multichannel retailing, each channel operates completely independently from each other – in fact, they may operate as if they are different