[Case Study] Can AI Emotional Recognition unlock new Performance Marketing opportunities?

Hey - have you heard about this new ChatGPT thing?

Well, this blog post is NOT about ChatGPT.

Now of course there is a lot to say about ChatGPT...

(and a lot more to dig into on the interesting & innovative ways marketers are using it right now. Btw, what's your favorite way to use it?).

...But we’ll leave it for another day.

Because generating content (whether text content with GPT or images/videos with DALL-E / Stable Diffusion / etc.) is just one area where AI is creating breakthroughs right now.

Today I want to talk about another, completely different AI use case we have been experimenting with over the past 12 months while seeing some very interesting results.

It’s all about using AI to get a powerful additional perspective on WHY certain ads work or don’t work… and maybe even predict how well your ads will perform before you even launch a campaign.

Let's dive in...

Emotion, Quantified.

Image source: https://www.nature.com/articles/d41586-020-00507-5

Our internal emotional state is in a constant state of flux.

And no matter how hard we try to hide it, we are constantly sharing this information with the world:

Because our moment-by-moment emotional state is literally written on our faces, whether we realize it or not.

You just need a trained set of eyes to see it.

Micro Expressions & Dr. Paul Ekman

So how exactly are we displaying our emotions to the outside world, without even realizing it?

Each moment we have very brief, involuntary facial expressions - called micro expressions - that reveal our true emotions. These last only a fraction of a second while being extremely difficult to detect without training.

Renowned psychologist Dr. Paul Ekman discovered this phenomenon, finding that there are certain facial expressions that are universal and are associated with specific emotions, such as happiness, sadness, anger, fear, and surprise. He saw that these expressions were so brief that they could only be documented by using methods like high-speed photography to capture and analyze these subtle facial movements.

Ekman's research isn't some kind of baseless pseudoscience.

The FBI, CIA, DHS & US Secret Service are just a few of many agencies that actively use Ekman’s microexpression strategies.

It takes an Ekman-trained microexpression expert months of full-time training to become skilled at seeing these types of patterns and reading faces.

But guess what else is a quick learner at pattern recognition?

You guessed it: a well-trained AI.

Check this out:

Here I am staring vacantly into the screen while being monitored by our facial scanning AI emotional recognition software.

You’ll see that our software is tracking my:

Emotional valence (positive vs. negative)
Emotional arousal (high vs. low)
And my eye movement

All this is thanks to mountains of manually labeled data from actual Microexpression experts, which has trained the model used here.

The result: automated & instantaneous emotion recognition & eye tracking.

Looking at a corresponding emotion graph you can see how much these emotion values change, moment by moment:

(Note on eye movement - Eye movement allows us to evaluate user attention & engagement, moment by moment - and even generate heatmaps, showing where people typically look in your videos, frame-by-frame. We will talk in more detail on this aspect in an upcoming post…)

Now AI emotional recognition is definitely not something new.

Companies like Affectiva have been doing pioneering work in the space for nearly 15 years.

But here’s the thing: these more established companies have been most interested in applying this tech to Fortune 500 & branding-focused spaces.

We on the other hand run a performance marketing business.

So even though seeing AI create these Tron-style image overlays is fun…

What we are actually obsessed with is trying to understand:

Can this type data give us insights to get measurably better results out of every dollar we spend on ads?

Well, can it?

Let’s take a closer look at one experiment we did, where we did an Emotional Recognition study to get a deeper understanding on what made creatives work / not work for one of our offers.

Experiment: What are the Key Emotions Behind a Winning YouTube ad intro?

We talk about it all the time: Just how important your YouTube ad intro or “hook” section is.

Remember those milestones:

At 5 seconds watched, the “skip ad” button pops up.

At 10 seconds watched, Google counts as an “engagement.” Higher engagement = better traffic costs for you.

The Setup:

We were having success with one offer where we had built and tested around 200 different YouTube ads, and spent over $10 million usd in YouTube adspend.

* We wanted to take a closer look at 9 of these ads (with a combined $8+ million usd adspend between them).

* Our winning ad in this group had over $3 million usd spent alone

* All ads in this cohort were the same “core” ad, with a different 5-20 sec initial intro on each

* Despite these similarities, performance between these ads varied widely, and we wanted to understand why.

The Experiment:

Audience: ~1000 unique participants, similar in age/gender/location to our best ad audiences (approx ~110 viewers per ad)

Environment: Home laptop environment, where we record & measure emotional / attentional response via their webcam (with consent).

The Questions:

Is there a specific emotional “signature” for our best performing YouTube ad intro? How does this compare to the other ads? What insights can emotional recognition tell us about what makes a high-performing YouTube ad intro?

The Results:

Before we move on, what is your guess on what emotion would drive the best YouTube ad intro?

Let’s look at the results through a few specific emotional metrics we measured:

Notice the 2 main axes:

Valence: Valence describes on a continuous scale whether the emotion is "negative“, “unpleasant” or "positive“, “pleasant”.

Arousal: Arousal indicates whether a person is more unaroused/relaxed or more aroused/excited.

So let’s start with Arousal.

Did our ads make viewers more excited or more bored? How about our best ad?

Looks like our best ad actually moved the audience the 2nd least vs. baseline in terms of arousal.

So our best ad = our 2nd most “boring” ad?

What’s going on here?

Let’s take another look at a different slice of data…

Not too much variation happening here on the sympathy-happiness axis it appears, let's move on...

Next: what about the emotion of “surprise?”

A shocking, suprising hook seems like it might work, right?

Unfortunately, no breakthrough with this. Our best ad is firmly the median value here, with as many ads with higher surprise value as there are lower:

Moving on, here’s where things start to get interesting, on the focus-anger axis:

Our best ad has the lowest value here, by far.

So our best ad has the intro that makes the viewer the least angry or frustrated? Interesting.

Let’s take a more comprehensive look at the data across all the ads in our study to see if there is indeed a correlation between performance and these values and ad performance generally:

(These are both statistically significant with P-values on each at 0.005 and 0.032, respectively)

To sum it up, for these ads we studied for this particular offer:

Focus-anger measurement correlates negatively with performance…
While the neutral, indifference-sadness measurement correlates positively with it…

Which means, in the case of this particular study on this particular offer…

The less annoying and less confusing (and even the more boring) the YouTube ad intro is, the better the performance.

(Again - for this particular offer with the data we have, at least)

Characteristics of Our Winning Intro:

What was different about our winning intro that resulted in such massively better performance?

(Maybe you are wondering so you can test it yourself...)

In this case, our best ad simply led with the price of the product, right away. (Before getting into the viewer’s needs, product features / benefits / etc.)

Does this type of intro always win, in every case?

Of course not. But we always try and at least test this approach, whenever we can.

A Predictive Model of Ad Performance?

What if you could predict an ad’s success, before you spend a cent on it?

Here we have a Machine Learning model predicting performance using the emotional data from the study:

Above you can see the 9 tested ads and their performance (normalized to a range of 0.0 to 1.0). The colored bars show the true value from the campaign data.

The gray bars show a prediction, each made by a model that was only trained on the emotion analytics results of the other 8 ads.

This process is repeated for every ad (this is called a leave-one-out cross
validation). So every time, the model uses data from 8 ads and then predicts the value for the 9th ad.

As we can see, the model is quite good at estimating the performance of an ad, simply based on the data from the emotional recognition model (NOT using any campaign data whatsoever)

The model clearly identified the actual breakthrough ad in this cohort, from the emotional data alone.

With enough offer-specific data, it could definitely be interesting to include this kind of predictive model in your ad creation flow.

What's next...

AI in marketing is a massive topic. And we can't wait to talk more about everything we have going on in this space "behind the curtain" at the moment.

But right now, if this has got you excited and you want help brainstorming how you can use AI to uncover your business' next big breakthrough...

If this sounds interesting to you? Let's chat.

(Just comment on this post)

This sort of thing is definitely not for everyone, and it's not magic...

...But it's guaranteed there are golden, game-changing insights sitting inside your ad account waiting to be uncovered & leveraged.

Again - go ahead and comment on this blog post if you want to brainstorm ways to apply AI insights like these to your own business.

- Sponsored -

Inceptly: the performance video ad team behind $950M+ in direct response revenue

Inceptly High-Performance YouTube Advertising Agency

Inceptly builds YouTube ad creatives & manages over 5M USD/month in YouTube ad spend for companies like ClickFunnels, Descript, MindValley, Advanced Bionaturals, Organifi, and many more.

Find our agency Inceptly here

Are you spending over $1k/day on ads and looking to scale your business with YouTube ads?

Schedule your free YouTube ad brainstorming call here:

inceptly.com/call

What are your YouTube ad questions?

In the meantime though, what questions do YOU have about YouTube ads?

Let us know in the Comments section below, and we’ll make sure to cover your question in an upcoming post.

Have a nice day!

The VidTao Team

VidTao.com

Are you spending over $1k/day on paid traffic and want to scale with YouTube ads? Schedule your free YouTube ad brainstorming call here: inceptly.com/call

VidTao.com is brought to you by Inceptly.com - High-Performance YouTube Ad Creative & Media Buying Agency Managing $5M/month+ in YouTube Ad Traffic.