This article uses a case history from MTV to examine the role that engagement with programming plays in the performance of embedded advertising. A standard technique for measuring emotional engagement with TV commercials, the Ameritest Picture Sorts, was used in an online study of 600 respondents to study audience engagement with the two-hour-long MTV Video Music Award show. Utilizing this simple nonverbal metric, along with conventional verbal metrics of advertising performance and brand perceptions, the authors show that programming has a “magnifier effect” on some advertising metrics, such as likeability and motivation—but not others, such as recall. Importantly, the strongest advertisements benefit more from the magnifier effect than weaker ones—reaffirming the central importance of creativity in advertising effectiveness.

Engagement has recently become a major topic of discussion in the advertising industry. In July 2005 things got serious when the Advertising Research Foundation, along with American Association of Advertising Agencies and the Association of National Advertisers, kicked off an initiative to put engagement front and center in the research and measurement process. In March 2006 the ARF unveiled a working definition, encouraging the industry to add findings to help clarify how engagement might become a measurement standard.

The ARF has defined engagement as “turning on a prospect to a brand idea enhanced by the surrounding context.” While this definition may undergo change as those in the industry contribute thinking on approaches to this new metric, the need for an expanded, more customer-centric view of advertising appears to be unanimous. Few would argue with the premise that an “engaged” viewer has more value than a viewer grabbing a beer and chatting with his spouse while ads play on the flat panel—despite increasing evidence that even “backgrounded” experiences have some subconscious effects, as outlined in Wendy Gordon’s recent paper “What Do Consumers Do Emotionally with Advertising” (2006).

It is important to recognize at this nascent defining stage that engagement is a complex subject and there are many ways to look at engagement with media. Advertisers learned long ago that it is a mistake to think of television commercials as one-dimensional, describable by a single metric such as day-after-recall. All now understand the need to measure multiple dimensions of performance—eg. attention-getting power, branding, motivation—in order to predict a commercial’s potential effectiveness. It would be equally simplistic to think of engagement with programming content one dimensionally. Here are just some of the different ways or contexts in which we might think of a viewer as being “engaged” with a television program:

1. Cognitive—to be focused and attentive to what is being shown and said at any given moment in the program;

2. Emotional—to be “hooked” and deeply involved with the unfolding storyline of the program;

3. Social—to watch a television show with other people;

4. Multi-platforming—to simultaneously interact with other platforms like the web or cell phones while watching a program.


Much work needs to be done in all areas in this fertile field of engagement. As an example, Young’s findings published earlier this year suggest a link between long-term advertising effects and those “branding moments” in film advertising that are strong both cognitively (are pre-consciously filtered for and remembered by viewers) as well as emotionally (elicit the strongest feeling). Young describes these peak moments where thought and feeling come together as “aesthetic emotion,” the creation of meaningful experiences for the consumer which is fundamental to the process of building a brand (2005). Findings reported in this paper will focus on the role emotion plays in engagement, though other forms of engagement with programming that were observed will be noted here and also explored more fully in future papers.

The part of the ARF definition of engagement that we are concerned with here is what it means for a brand idea to be “enhanced by the surrounding context.”

For those who depend on advertising to fund the creation of content—namely television networks—exploring exactly what it means to call an audience “engaged,” and exactly what happens when it happens, is vital. MTV’s desire to more deeply understand their viewers’ engagement with their programming, and how that engagement impacts embedded advertising, resulted in them joining with Ameritest to study one of their most highly-viewed programs: the annual Video Music Awards (VMA)—adding to the growing body of knowledge on engagement and advertising.

It was important to MTV to use the same tools to measure engagement with programming that their advertisers used for the study of their advertising—not only to contribute to the work taking place on engagement, but to employ validated measurement tools that would be accepted and trusted by MTV’s advertisers. For this study MTV chose one of a variety of approaches being used today by major advertisers to measure emotional engagement with their television commercials, the Ameritest Picture Sorts™.

Literature Review


Many contributions have been made toward a better understanding of engagement with advertising, including those findings offered by participants on the joint committee formed in 2005 by the ARF and the AAAA to study emotional response in advertising. Many methodologies have been used to contribute, from those experimental approaches that employ physiological measures such as EEG and facial response, to more traditional ad testing methodologies that continue to work with dial meters and verbal rating statements. Mark Truss of JWT advertising and Alice Sylvester of FCB advertising summarized some of the different approaches to measure emotional engagement with beer advertising in a presentation to the 2005 Account Planners conference, as shown in Exhibit 1.

One thing that is interesting about the work so far, though perhaps not surprising, is that each of these different commercialized approaches to measuring emotional response to advertising drew somewhat different conclusions as to which of a set of different beer commercials produced the strongest emotional response. For example, three of the approaches (Gallup and Robinson’s facial response, Answer Stream’s EEG measures and Ameritest’s Picture Sorts) came down on the side of the famous and “edgy” (i.e. somewhat polarizing) Budweiser “Whassup!” campaign as generating the most emotion, while three others (Ad Sam’s emotional ratings, TNS emotional verbatim coding, and MSW’s dial meter) reported that two other, more conventional Heinekin or Bud Light commercials were stronger ads.

These divergent findings suggest the need for additional work to take these different operational measures of emotion and validate them back to accepted performance metrics, or “report card measures,” that advertisers use to make commercial airing decisions: attention, recall, branding, liking, motivation or persuasion.

Indeed, there is ongoing debate about the role that emotion plays in some of these metrics. Kastenholz et al (2004) analyzed the internal structure of a sizable database of packaged goods commercials and found that emotion played no significant role in day after recall scores, which picture sort analysis determined was primarily driven by the semantic content of an ad. More recently, Mehta and Purvis (2006) suggested the opposite to be true, finding a high correlation between the likeability of an ad and the recall scores of ads in their database—but without addressing directly the question of whether or not the focus of the advertising was semantic content, e.g. product news, or aesthetic content, e.g. brand emotions. One fruitful area of further research would be to use these new engagement tools like facial response to explore the relationship between emotion and day-after recall in greater depth.

EEG research represents a perennially intriguing way to measure emotional engagement, especially given the great strides being made these days by medical researchers in understanding the physiology of how the brain works. But operational development is likely to proceed slowly given the complexity of the task. There is more than one kind of brain wave and much remains to be understood about the emotional “signal” contained in various combinations of brain wave patterns. For instance, applying a brain wave technology developed by NASA for the study of astronauts to advertising research, Young (2002) found that the particular algorithm of brain wave activity being used by NASA to study engagement (beta /(alpha + theta)) responded more to the semantic or conceptual content, not the emotional content, of the television commercials being studied

An issue that many advertising creatives might have with some approaches to measuring emotional engagement is the focus on the one dominant emotion being produced by a commercial. Students of film, like the famous Hollywood writer/lecturer Robert McKee (1997), teach that engaging film-making is tied to the change in emotions from the beginning to the end of a scene. Good storytelling, he reminds us, is how the audience is moved by film.

One interesting finding about these different research techniques to measuring engagement (relevant to understanding the study reported here because dial meters are frequently used to study movies and programming content as well as TV commercials) is that no significant correlation has been found between the self-report dial meter systems used by companies like MSW and Millward Brown and the picture sorts used by Ameritest. This lack of correlation was seen during the course of the ARF committee’s work reported above, replicating other proprietary studies conducted by our clients. The reasons for this difference are beyond the scope of this paper and will be the subject of a future article. But it should serve as a caveat emptor that while many researchers claim to be measuring “emotional engagement” that does not mean that we are all measuring the same thing.

Work of the committee on TV commercials has naturally added to the more general work being done on engagement, as emotion is the key driver of a prospect being “turned on,” to use the ARF’s working definition. As part of the Committee, our own study of longer format advertising film—the “branded entertainment” of the famous BMW online films (Shea 2005) has added some additional findings that support our experience of how emotion in advertising has always worked—namely, it is not enough to use film to simply evoke an emotion of any kind, but rather specific and relevant emotions consistent with brand values need to be created and integrated into the storyline of the film in the service of the brand. In terms of the ARF engagement definition, you must do more than turn on a prospect, you must turn on that prospect to a brand idea.


Research was conducted on-line among 640 respondents representing MTV’s core audience of 13-24 year olds, male and female, who regularly watch MTV. The study was divided into a test cell of n=480 of those regular MTV viewers who watched the Video Music Awards (VMA) program and a control cell of n=160 regular MTV viewers who did not watch the VMA program. The test and control cells were matched demographically for age, gender and ethnicity.

The study was conducted on the internet so that respondents could be exposed to visual stimuli from both the program and embedded advertising, as well as questioned on program reactions, viewing context, multi-platform behavior, brand preferences and awareness and attitudes toward the advertising. The thirty-minute self-administered survey was fielded three days after the VMA aired in September, 2005, and was in field for one week.

Recognition and response measures were obtained to fifteen commercials embedded in the program, using six key frames as a recognition stimulus—comparable to the standard approach being used today by major advertisers doing on-line advertising tracking research.


Two picture sorts were conducted on the two-hour program itself, as shown in Exhibit 2. The first visual sort, the Flow of Attention (FOA), is a cognitive measure of pre-conscious filtering—what the mind allows into memory. The second visual sort, the Flow of Emotion (FOE), measures emotional response to those images that were recognized. Therefore, the Flow of Emotion can be viewed as the result of a two-step filtering process. In this part of the interview respondents sort through a randomized set of visuals taken from the test film and report on whether or not they remember the image (the FOA) and, if that image is remembered, how they felt about it on a five-point scale from high positive to high negative (the FOE). This paper will focus on the Flow of Emotion in order to contribute findings on how emotion contributes to engagement.

For readers unfamiliar with this widely-used commercial measurement technique, we should make a couple of points regarding the appropriateness of using the visual sorting process for understanding programming content.

The idea that still images can be used to capture the emotions of a film is well established in Hollywood. In describing his work habits as an editor of such films as Godfather Part II, Apocalypse Now, Ghost and The English Patient, Walter Murch (2001) talks about how he creates an emotional language for the film he is working on. From the raw footage he takes a representative still from each set-up and arranges them in panels on the wall as a visual vocabulary for the emotional language he is working with:


“…the most interesting asset of the photos for me was that they provided the hieroglyphs for a language of emotions. What word expresses the concept of ironic anger tinged with melancholy? There isn’t a word for it, in English anyway, but you can see that specific emotion represented in this photograph.”

Or, as stated by Mary Corliss, Assistant Curator in the Department of Film and Media at the Museum of Modern Art in New York (2006):

Francois Truffaut acknowledged the potency of the still image when he ended his first feature, “The 400 Blows,” with a freeze frame of his young hero. It captured Antoine Doinel (Jean-Pierre Leaud) in a moment in time, his future uncertain, his face seemingly asking “Now what?” at the end of the first turbulent chapter of his experiences. That’s what film stills do. They freeze the emotion and excitement of an actor, a scene, a film, and era; they are the pin through the movie butterfly that somehow gives this lovely, ephemeral creature lasting life. Stills distill; stills preserve.

For research purposes, the Picture sort technique builds on these ideas. Using a non-verbal or right-brain scanning and sorting process, respondents sort through still images to reconstruct their visual—and emotional—experience of the film or video, without having to resort to words.

One of the keys to the technique is how finely the film is “thin sliced” into still pictures. The idea of thin slicing was described by the writer Malcolm Gladwell (2005) in his book Blink to describe the ability of the mind to unconsciously or pre-consciously process experience very rapidly:


“‘Thin-slicing’ refers to the ability of our unconscious to find patterns in situations and behavior based on very narrow slices of experience.”

The stills that are used in the picture sorting process are not chosen according to clock time or uniform increments (which is one of the main differences from dial meter measurements), but are taken to represent perceivably different, meaningful slices of the film experience. That is why there is no fixed or pre-determined number of pictures to be used in the sorting process. The number of pictures, the partitioning of the film experience, has to be tied to content. In a sense, what we are doing is taking a stratified, not a uniform, sample of the flow of visual information in the film.

For thirty second television commercials, the film is sliced very thinly, because it turns out empirically that that level of granularity is very actionable for the purposes of re-editing film to improve commercial performance. For a two-hour movie or television program, the slices will be thicker. For the purpose of capturing the context effects of the VMA program 66 stills were sufficient.


One of the interesting observations that can be made about comparing the seismographic-looking flow graphs from film of different lengths is that within a certain range human perception—the search patterns of our conscious attention and the vibrations of our emotions—appears to scale fractally. Complexity scientists have repeatedly pointed out that if you were to zoom in on a picture of the coastline of England, changing scale by changing the length of your yardstick, the overall pattern would look the same. Similarly, if you were to look at the unlabeled picture sort graphs of a two-hour movie, a seven-minute online film, or a thirty-second commercial you might have a difficult time telling which is which. What this suggests is that, on the level of film syntax and dramatic structure, thirty-second commercials and two-hour movies operate on the same principles of the human mind—the problem of the emotional engagement with advertising and the problem of emotional engagement with content is really the same research problem.

The Engaged Viewer

The cliché about “when you are holding a hammer every problem seems like a nail” serves as a cautionary tale to researchers who can forget to see questions beyond the tools they most use to answer them. In the case of this study, we did not begin by first looking at the measurements we had in hand but instead worked backwards from the desired behavior of the audience. From a programmer’s standpoint, what would be the essential difference between an engaged and a non-engaged viewer?

In this age of TIVO, the hope and the expectation is that an engaged viewer would stay to the end and watch the entire program, while the non-engaged viewer would not.

Thus, we began our analysis by looking for contextual variables that are correlated with full-program viewing.

Social Context:


Viewing the award show as a social experience—that is, watching the program with others—contributed to the continuity of whole-program viewership, as shown in Exhibit 3. Those who watched with someone else were significantly more likely to watch the whole show. This suggests that social engagement might have a multiplier effect on the raw audience ratings for highly social programs such as the VMA.

Multi-platform Context:

Exhibit 3 also shows that for many the VMA was a multi-platforming experience, as well. For a quarter of viewers, accessing alternative content online or through video on demand (VOD) was part of the event. As seen with the social component, this multi-platforming—particularly likely among younger audiences of television programs today—again resulted in watching more of the program. This form of engagement with technology certainly has implications for the new world of integrated, 360 degree-Ad surround marketers.

In an old world view, multi-platforming could be construed as not paying attention—the programming nothing more than background while viewers navigate the web. However, the Flow of Attention for the VMA demonstrates this was not the case, as attention overall was good throughout the program. This reminds us that engagement is multi-dimensional and measuring it, most especially for today’s audience, requires cognitive measures like the Flow of Attention as well as the emotional measures that are more the focus of this paper.

Emotional Context:


Exhibit 4 demonstrates that those who watched the whole show were also more emotionally engaged, as measured by the Flow of Emotion. The Most Engaged curve is not simply the Least Engaged curve shifted upwards; these audiences are also responding to different moments in the program.

In terms of face validity, at least, this measure of emotional engagement with the program is tied to a reported behavior the programmers and marketers care about.

The Flow of Emotion

A look at the Flow of Emotion for the VMA offers highly specific insights on those moments in the program that resonated emotionally for viewers of this program with a history of pushing the envelope. The results of the Flow of Emotion for the VMA tracked with both MTV’s expectations as well as industry press.


Exhibit 5 demonstrates the emotional structure of the two-hour award program, showing peak emotional moments, both positive and negative. Footing with MTV’s extensive knowledge of contemporary music culture, the strongest positive emotional peaks in the show affirmed the popularity of such stars as Kelly Clarkson, Alicia Keys and Destiny’s Child. Polarizing moments that resulted in strongest negative emotion for many viewers were also no surprise: R. Kelly with his legal troubles; P. Diddy wearing a controversial t-shirt; Eva Ligouri and her questionable humor regarding hurricane Katrina; and the ever-polarizing Paris Hilton were all negative emotional peaks.

The Emotional Magnifier Effect


The Flow of Emotion was next used to explore the interaction between emotional engagement with program content and the embedded advertising. Using the Flow of Emotion data, two groups were created to view response to the embedded advertising. The VMA audience was divided into two equal-size groups based on level of emotional response to the stills sorted from the VMA program, averaged across all stills sorted. This resulted in two groups: the “Most Engaged” and “Least Engaged,” as shown in Exhibit 6. To test whether or not the difference between the two groups was just a halo effect, we ran a correlation between the two time series data. We found them to be only moderately correlated (r2=37%). Hence, the Most Engaged group is not only responding at different levels, but is also responding to different elements in the program. The Picture Sorts allows us to identify the different moments that separate the two groups.


Males and females, and younger and older, were equally engaged by the program, as demonstrated in Exhibit 7. This suggests that the differences described below which are correlated with this measure of program engagement are not explainable by differences in the demographics of these two halves of the audience.


A stronger level of emotional engagement with the VMA program was found to be related to a stronger response to the embedded advertising—though the effects were not the same for all measures or all the advertising.

The Most Engaged did not recall the ads any better than did the Least Engaged, but they did report higher levels of liking for the ads, particularly among the most likeable ads, as shown in Exhibit 8.

The difference in respondents’ ability to remember the advertising was a non-significant 5.4% on average. Also of note, the ads with the highest recall scores did not show a difference when compared to ads with a low recall score. However, since we did not have knowledge of prior exposure to any of these ads before appearing on the VMA we cannot make any statements about the relative strength of these ads on this particular metric. Nevertheless, this finding appears to go along with earlier research (Kastenholz, 2004) which suggests that emotion is not a major factor in the recall of TV commercials, either from within the ad itself or from the surrounding program.

Ad Liking, however, was significantly impacted by emotional engagement with the program, with an average 24.6% stronger response to the advertising among the most engaged compared to the least engaged. (See Exhibit 8) However, inspection of individual ad performances reveals that not all ads benefit from the effect equally.

These differences in ad liking are not due to differences in the brand being advertised, since ads for the same brand appear in both the top and bottom halves of the list. In theory, this is a factor which could have contributed to the difference—the appropriateness of the advertising to the program content. The fit between the brand and the program is certainly one of the major factors which a marketer takes into account when buying media. You would not, for example, expect to see an IBM commercial on South Park, nor would you expect to see an Axe body-spray ad on Sixty Minutes. On judgment, however, all of the brands and ads that were included in the study were quite MTV-appropriate.

One possible explanation of the magnification effect on the likeability of an ad embedded in an engaging program is the occurrence of some kind of emotional resonance—the effect of two “voices” singing together in concert, or counterpoint, acting as an emotional amplification that occurs when the content, mood or tone of a commercial builds on what came before. For the ongoing study of engagement, this would be a fruitful area for additional research: to identify the emotional structures and characteristics of program content and advertising executions that work particularly well together.

But perhaps the most important observation here is that those ads that are already most liked see more of a lift; weaker ads do not experience as strong a magnifying effect. This suggests that creative excellence matters on both ends and is an important variable in how the surrounding emotional context impacts advertising. Good ads are strengthened by the emotional surround of the program content, but good program content can’t compensate for a weak advertising concept.


Finally, and most importantly, an impact on purchase intent and many brand ratings was also demonstrated, as shown in Exhibits 9 and 10. Thus we complete our logic that by starting with a behavior that programmers care about—staying to the end of the program—one arrives at a behavior that sponsors care about—buying the brand. In terms of intended shopping behavior, those who watched the VMA program were motivated to shop the sponsors at higher levels than those in the control because of the embedded advertising. However, the effect was significantly greater for those audience members most emotionally engaged with the program. The same was true for sponsor brand ratings. The Most Engaged showed the highest levels against the control group. This is what we call the program “magnifier effect.”


This is exactly the kind of effect anticipated by the ARF; prospects would be turned on to a brand more strongly when advertising is enhanced by emotional engagement with the surrounding context. This finding has been subsequently replicated in other studies conducted on MTV programming. But much work remains to be done in terms of understanding the kinds of program content, and the types of advertising, that will benefit the most from this emotional magnifier effect.


This study helps build on the work currently being done to further define and create metrics for engagement. These findings suggest that a strong emotional engagement with programming can indeed magnify engagement with embedded advertising for some measures, such as liking, motivation, and perceptions of the brand—though it impacts recall of advertising at insignificant levels.

Finally, the finding that not all embedded advertising benefits equally from a strong emotional engagement with surrounding context speaks to the importance of good creative in the advertising executions themselves. The ability of the media vehicle to emotionally engage cannot substitute for excellence in both media and the message.


Alwitt, L. “EEG Activity Reflects the Content of Commercials.” Psychology Measures of Advertising Effects: Theory. Hillsdale, NJ: Erlbaum Assoc., 1989

Barnes, J. “The Complete Works of Aristotle: The Revised Oxford Translation.” Princeton, NJ. Princeton University Press, 1971

Braithwaite, A. and Swindells, A. “Qualitative Advertising Research.” Proceedings of the Market Research Society Conference, 1995.

Ebert, R. “The Great Movies.” New York, NY. Broadway Books, 2002

Gordon W. “What Do Consumers Do Emotionally with Advertising?” Journal of Advertising Research, March 2006.

Kastenholz, J., Kerr, G. and Young, C. “Does Day-After Recall Testing Produce Vanilla Advertising?” Admap, June 2004.

McKee, R. “Story: Substance, Structure, Style and the Principles of Screenwriting.” NY, NY: Harper Collins Books, 1997

Metha, A., and Purvis, S.C. “Reconsidering Recall and Emotion in Advertising.” Journal of Advertising Research, March 2006.

Moles, A. “Information Theory and Esthetic Perception.” Joel Cohen, Trans. Urbana, IL. University of Illinois Press, 1968.

Murch, W. “In the Blink of an Eye.” Beverly Hills, CA. Silman-James Press, 2001

Rothschild, M. and Hyun, Y.J. “Predicting Memory for Components of TV Commercials from EEG.” Journal of Consumer Research, 1989.

Shea, A. “BMW Films: From Luxury Car to Movie Star: Where Advertising and Film Converge.” Proceedings of the AAAA Account Planning Conference, August 2005.

Truss, M. and Sylvester, A. “New Thoughts on Measuring Emotional Response to Advertising: The AAAA/ARF Study.” Proceedings of the AAAA Account Planning Conference, August 2005.

Young, C. “Aesthetic Emotion and Long Term Ad Effects.” Admap, April 2006

Young, C. “Brain Waves, Picture Sorts, and Branding Moments.” Journal of Advertising Research, April 2002.

Zaltman, G. “How Customers Think: Essential Insights into the Mind of the Market.” Boston, MA. Harvard Business School Press, 2003