Reading the Consumer’s Mind: What Your Brain Knows About Heineken (and Durex)

What do you really feel when you watch an ad? Could you even tell me if you wanted to? What happens when we stop asking people these questions and start looking inside their brains instead?

Let’s start with a little experiment. Think of Heineken: not the logo, but the picture that pops up in your mind when you think of the brand. What do you see? Now do the same for Kellogg’s, then for Microsoft and, while we’re at it, for Durex.

Chances are you saw something like a group of friends at a party, a family at the breakfast table, someone working at a desk in an office and.., well... This is not a coincidence. Companies spend fortunes on getting exactly those pictures into your head: Heineken wants you to think of young people having a good time, Kellogg’s wants you to think of a loving family, and Microsoft would like you to think of a productive working day. That picture in your head (marketers call it the brand image) is in many ways what brands are really selling.

Now suppose you are the marketing director at Heineken, and you have just spent a few million on a new campaign. How do you find out whether the picture you wanted actually made it into people’s heads? The obvious answer is to ask them, and that is exactly where things get tricky.

In my lab we try a different route. We have people think about brands, watch commercials or look at print ads, and in the meantime we record what is going on in their brains. Then we try to read their minds: not in the science-fiction sense, of course, but we can get surprisingly far. So how does that work, and what does it tell us that people can’t (or won’t) tell us themselves?

Why we can’t just ask

Asking people how they feel about an ad, or what they associate a brand with, seems like the easiest thing in the world, and marketers do it all the time, in surveys, focus groups and copy tests. But there are quite a few problems with it.

First, people have surprisingly little access to what goes on in their own minds, because a lot of what drives our behavior happens below the level of conscious experience. Ask people why they bought something and they will happily give you an answer; it’s just not necessarily the right one.

Second, even when people do know what they feel, they often find it hard to put into words. Try describing exactly how the last commercial you saw made you feel. Pleasant? A bit excited? Mildly annoyed? For most of us, that is about as far as it goes.

Third, people don’t always want to tell you. People are quite happy to tell you what they associate with McDonald’s, but ask them what they associate with Durex and the room goes rather quiet, since nobody wants to be the one in the focus group who blurts out the first thing that comes to mind.

Fourth, when we ask people about an ad, we usually ask them afterwards. So what we are measuring is not what they felt while watching, but what they remember feeling, and memory is a notoriously creative storyteller.

And finally, feelings change from moment to moment. A 30-second commercial can make you curious, then amused, then bored and then moved, all in the time it takes to make a cup of coffee, and it is often exactly those ups and downs that make or break an ad. But try asking someone how they felt at second 12 of the commercial they just saw. Good luck with that.

So if asking doesn’t work all that well, what does?

How to read a mind (sort of)

This is where neuroscience comes in. With brain scanners we can follow what happens in someone’s head while it is happening, without having to ask them anything at all. We mainly use two tools for this. With EEG (electroencephalography) we place electrodes on the scalp and measure the electrical activity of the brain, millisecond by millisecond. With fMRI (functional magnetic resonance imaging) the participant lies in a large scanner that tracks changes in blood flow in the brain, which tells us which brain areas are working hard.

For a long time, the main question neuroscientists asked with these tools was where in the brain something happens: which area “lights up” when you feel fear, see a face or think about money? Those are interesting questions, but marketer typically couldn’t care less. If you are selling beer, you don’t need to know which brain area is involved; you want to know what people feel or think when they see your ad.

And that is precisely the question that a newer approach tries to answer. Emotions, thoughts and memories are not neatly tucked away in one particular spot in the brain, but show up as patterns of activity spread out across many areas at once. Using machine learning (for the connoisseurs: multivariate pattern analysis, or MVPA), we can teach a computer model to recognize those patterns.

You can think of it a bit like learning to recognize a song. The first few times you hear it, someone has to tell you what it is, but after a while you only need to hear a couple of notes to know. In the same way, we first show people things of which we already know what feelings they evoke, so the computer can learn what the brain pattern for, say, “excited” or “pleasant” looks like. Then we show them something new, like an ad, and let the computer tell us which pattern it recognizes.

So what happens when we try this with real ads?

Exciting ads: noticed, but not loved

To find out, my colleagues and I ran an experiment with EEG (Eijlers et al., 2020). Thirty-one participants first looked at 100 photos from a standard set that psychologists use to evoke emotions, the International Affective Picture System (IAPS for short). Some of these photos are pleasant (puppies, happy families) and some unpleasant (accidents, snakes), some are very exciting and some are rather dull. Because we know which feelings these photos evoke, we could use them to teach our computer model what an “excited” brain looks like.

Then the same participants looked at 150 real print ads, for everything from cars to cosmetics, and we let the computer estimate, for each ad, how much excitement (or arousal, as psychologists call it) it evoked in their brains.

Of course, what matters to an advertiser is not what happens in the brains of 31 students in a lab, but whether the ad works out there, in the real world. Luckily, the same ads had also been rated by a large panel of 1,260 consumers, who indicated how much each ad stood out and how much they liked it. So we could check whether the excitement we measured in the brains of our small group says anything about how the population at large responds to these ads.

It does, but not quite in the way you might expect. Ads that caused a lot of excitement in the brain stood out more for the consumers in the panel: they got noticed, which is exactly what advertisers want, right? Well, partly. Because these same exciting ads were also liked less!

So should you make your ads as exciting as possible? Not necessarily: grabbing attention is one thing, but if people end up disliking your ad, you may not have done your brand any favors. It must be said that these effects were modest, and brain activity certainly doesn’t explain everything about an ad’s success. But it is a trade-off you could easily miss if you only ask people whether they like an ad.

From photos to film

Print ads are nice, but these days advertising is mostly about video, and video is a different beast altogether. Where a photo evokes one feeling, a video takes you on a rollercoaster ride, with the emotions shifting from one second to the next.

So in a next study, we wanted to know whether our approach would also work for dynamic images (Chan et al., 2020). Twenty-eight participants again looked at IAPS photos, this time in the MRI scanner, and in a separate session they watched 18 movie trailers, each between one and two and a half minutes long. Again, we trained a computer model on the brain patterns evoked by the photos, and then let it loose on the trailers, second by second.

What we found is that the computer, trained only on still photos, could follow the emotional ups and downs of a trailer as it unfolded. We compared its estimates with how excited a separate group of 51 people said they felt while watching the same trailers, moment by moment, and the two tracked each other nicely (for how pleasant the trailers felt, the match was weaker). So the feelings evoked by a photo of a puppy or a snake look enough like the feelings evoked by a Hollywood trailer for the brain to give them away.

The first three seconds

So far, we have only looked at emotions. But there is a lot more going on in your head when you watch an ad: you perceive images and sounds, you pay attention (or not), you remember things, and you try to figure out what the people on screen are thinking and feeling. All of these may matter for whether you end up liking the ad.

The problem is that to decode all those different processes, we would need to train our computer model on each of them separately, which would take forever. Fortunately, there is a shortcut. Over the past decades, neuroscientists have published thousands of brain imaging studies on just about every mental process you can think of, and a database called Neurosynth has collected the results of more than 14,000 of them. For every concept, from “emotion” to “memory” to “attention”, it can tell you what the typical brain pattern looks like. So instead of training the computer ourselves, we can compare the brain activity evoked by an ad with this enormous library, and see which mental processes it resembles most: all of them at the same time, and moment by moment.

That’s what we did in a large study with 113 participants in the US and the Netherlands, who watched 85 video ads in the MRI scanner (Chan et al., 2024). For every second of every ad, we decoded which mental processes were active, and then checked which of those processes predicted how much people liked the ad (see Figure 3).

Emotions mattered, of course, but not throughout the ad. Emotional responses were important right at the start: within the first three seconds they already predicted whether people would like the ad, after which their influence gradually faded. In other words, if an ad doesn’t touch you in the first few seconds, it has a problem.

Then something else took over. The strongest predictor of all turned out to be what psychologists call mentalizing: thinking about what other people think and feel. What do the characters on screen want, why are they doing what they are doing, and how is this going to end? Mentalizing kicked in a few seconds into the ad and then stayed important until the very end. Put simply, people like ads that tell a story about people.

And at the end of the ad? There, the processes that predicted liking had to do with perception, with simply taking in what you see and hear, so ads that ended on something for the senses (a zoom, a slow-motion shot) were liked better. But ads that made people think hard, engaging the brain’s machinery for effortful control (executive function, in psychology speak), were liked less. That won’t come as a surprise to anyone who has ever tried to read the small print at the end of a car commercial.

Interestingly, these brain signals were not just telling us something about the people in our scanner. When we used them to predict how much a completely different group of people liked the ads, they did a better job than our own participants’ liking ratings. The brain, it seems, knew something that people’s own answers did not.

Brands in the brain

Let’s go back to Heineken. Ads are one thing, but the bigger goal of all that advertising is to build a brand image: a set of associations that pops up in your head whenever you see the logo. Can we read that from the brain as well?

There is a good reason to think that we can. When you imagine something, your brain uses much the same machinery as when you actually see it, which is why a computer trained on the brain patterns evoked by real pictures can tell, at least roughly, what someone is picturing in their mind’s eye.

So we asked participants to come up with their own mental image for each of 14 well-known brands, including Apple, Disney, Heineken, Kellogg’s, Microsoft, Red Bull and, yes, Durex (Chan et al., 2018). Then, in the MRI scanner, we showed them the brand logos and asked them to call up that image again. We also showed them more than a hundred photos of four kinds of social situations: people working in an office, partying with friends, being intimate with a romantic partner and spending time with family. By comparing the brain patterns evoked by the brands with those evoked by the photos, we could build a kind of brain profile for every brand: how much does this brand resemble “party”, how much is it like “family”, how much like “work” and how much like “romance”?

Many of the results will sound familiar from our little experiment at the start (see Figure 4): Heineken looked like a party, Kellogg’s looked like family and Microsoft looked like work. But there were surprises, too. Heineken, for instance, also looked quite a lot like work (Friday afternoon drinks, perhaps?), and it was clearly considered less sexy than Red Bull. And some brands, like Heineken, Disney and Durex, evoked very similar brain patterns across different people, which is what a strong brand looks like: everybody has the same picture in their head.

What’s more, these brain profiles turned out to be useful. When two brands had very similar profiles, like Apple and the headphone brand Beats, people said they would be more interested in a product made by the two together; when the profiles were very different, like those of Kellogg’s and Durex, not so much (I’ll let you picture that one yourself). So if you are looking for a partner brand, it may well pay to check whether you live in the same part of the consumer’s head.

We were not the only ones to try this, by the way. A team at Berkeley used a similar approach to read the “personality” of brands from the brain, such as whether people see a brand as sincere, exciting or rugged (Chen, Nelson & Hsu, 2015), and that worked as well.

“Isn’t this all a bit creepy?”

When I talk about this research, I usually get a couple of questions. The first one is some version of: isn’t this creepy, and can companies now read my mind?

The honest answer is no, or at least not without your full cooperation. All of this research takes place in a lab, with participants who signed up for it and who are lying still in a scanner for an hour or so. The computer can only recognize the patterns it has been trained on, and it needs the participant to play along. Nobody is going to scan your brain while you are walking down the supermarket aisle, and what we decode is fairly coarse: excited or not, party or family, but certainly not your PIN code.

“Can we really not just ask people?”

The second question is: if all this is so much trouble, why not just ask people? That’s a fair point, and in a way we actually do, since our decoders are built on asking: we train them on photos that we know evoke certain feelings because people have told us so, over and over again, in many studies.

But the brain adds things that asking can’t. It tells us how people feel while they are watching, second by second, rather than how they remember feeling afterwards, and it gets around the awkward silence in the Durex focus group. And, as the video ad study showed, brain responses from a small group of people can sometimes predict how a much larger group will respond better than the small group’s own answers can.

“Is it worth the money?”

The third question usually comes from people in industry: is it worth it? An MRI scanner is not cheap, and studies like ours take months to run.

Here I think some modesty is in order. The groups of participants in these studies are small, from a few dozen to just over a hundred people, and the relationships we find are real but modest: brain measures add information, but they don’t replace everything else. EEG is a lot cheaper than fMRI, but it may also tell you less. So for the local bakery, this is probably not the way to test the next flyer. But for a campaign worth millions, or for understanding why some ads work and others don’t, it may very well be.

So what should advertisers do?

That said, our research does give some concrete pointers, and most of them don’t require a scanner at all, because us neuromarketing researchers did the hard work for you:

• Grab them early, and emotionally. Emotional responses in the first few seconds predict whether people will like your ad, so don’t save the good stuff for the end.
• Tell a story about people. Getting viewers to think about what the characters want and feel was the strongest predictor of liking, even in short ads.
• Don’t make them think too hard at the end. End on something for the senses, not on a wall of text.
• Remember that exciting is not the same as liked. An ad that gets noticed is not necessarily an ad that people like, so know which one you need.
• Pick your partners carefully. Brands that occupy the same place in people’s heads make better partners.

Back to Heineken

So, what did you see when you thought of Heineken at the start of this piece? A party, probably. And some of you, if you’re honest, may also have thought of Friday drinks at the office, even if you might not have mentioned that if I had asked you. But don’t worry: your brain would have told me anyway!

References

• Chan, H.-Y., Boksem, M. A. S., & Smidts, A. (2018). Neural profiling of brands: Mapping brand image in consumers’ brains with visual templates. Journal of Marketing Research, 55(4), 600–615. https://doi.org/10.1509/jmr.17.0019
• Chan, H.-Y., Smidts, A., Schoots, V. C., Sanfey, A. G., & Boksem, M. A. S. (2020). Decoding dynamic affective responses to naturalistic videos with shared neural patterns. NeuroImage, 216, Article 116618. https://doi.org/10.1016/j.neuroimage.2020.116618
• Chan, H.-Y., Boksem, M. A. S., Venkatraman, V., Dietvorst, R. C., Scholz, C., Vo, K., Falk, E. B., & Smidts, A. (2024). Neural signals of video advertisement liking: Insights into psychological processes and their temporal dynamics. Journal of Marketing Research, 61(5), 891–913. https://doi.org/10.1177/00222437231194319
• Chen, Y.-P., Nelson, L. D., & Hsu, M. (2015). From “where” to “what”: Distributed representations of brand associations in the human brain. Journal of Marketing Research, 52(4), 453–466. https://doi.org/10.1509/jmr.14.0606
• Eijlers, E., Smidts, A., & Boksem, M. A. S. (2020). Measuring neural arousal for advertisements and its relationship with advertising success. Frontiers in Neuroscience, 14, Article 736. https://doi.org/10.3389/fnins.2020.00736