RoughingIt_cover

Introduction

The typical industry rationale for pre-testing commercials at a rough level of finish can be summed up in five words: screen out bad ideas economically. In short, rough pre-testing has been used as a “go/no go” early warning system to minimize waste and limit the time and expense allocated to fully produce an execution. Put another way, if a rough commercial doesn’t score well, then go no further—with the assumption, of course, that the research score of the animatic is predictive of its score as a fully-finished commercial. Testing roughs is economical considering that the average cost of an animatic is roughly a tenth of the cost of a fully-finished commercial.

Unilever is one of many advertisers realizing cost efficiencies in this business process by learning about the creative at rougher levels of finish at the outset (e.g., using animatics, photomatics, stealomatics, etc.). As client and agency converge in their satisfaction with a creative idea at this more preliminary inexpensive level, more successful efforts are taken to higher levels of costly full-production.

In general, previous research on the validity of rough testing has focused on the comparability of scores for rough and finished ads, demonstrating the predictive value of working with ads at lower levels of finish in order to filter creative ideas in the early stages of the advertising development process. However, from our perspective, this is only half the value of pre-testing. The other important benefit of working with rough commercials is to use the diagnostic information from the test in order to optimize the performance of the final film, a subject about which much less research has been published.

RoughingIt_summary

As Peter Senge points out in his book on learning organizations, The Fifth Discipline, “The almost total absence of meaningful “practice” or “rehearsal” is probably the predominant factor that keeps most management teams from being effective learning units.” Accordingly, in applying this management philosophy to the development of effective advertising, our practice is to rehearse the creative in rough form first.

Both academic and industry researchers have found that there are more similarities than differences in the general effectiveness of rough and finished versions of the same execution on such performance measures as attention, motivation, semantic communication, and emotional communication. The implicit theory underlying these studies is that roughs should be used to filter rather than optimize.

We were unable to locate any published research to date on the optimization function of roughs. Nor could we re-interpret or re-analyze published research so it would shed light on this issue. Accordingly, what follows is an empirical analysis of the benefits of “rough rehearsal” from the Ameritest database of Unilever pre-test results.

Current Research by Ameritest and Unilever

The primary objective of this research was to assess the difference in the branded breakthrough power of ads that went directly into final production without further testing, versus those that were rehearsed first in rough production.

RoughingIt_callout

Specifically, we compared commercial performance scores across the following four mutually exclusive groups which, taken together, represented the complete testing history for a specified set of Unilever brands over a relatively short, recent time period:

• Group 1: Ads that had been tested only in animatic form, with no subsequent testing of the final film for those ads that were selected for production [More than 50 ad pre-test results];

• Group 2: Ads that had been initially tested in animatic form and were eventually re-tested as final film commercials. These data represent the performance of these ads as animatics, early on in their developmental lifecycle [n=14 ad pre-test results];

• Group 3: The same ads in group two that had been initially tested in animatic form and were eventually re-tested as final film commercials. These data represent the performance of these ads as final film commercials, later on in their developmental lifecycle [n=14 ad pre-test results];

• Group 4: Ads that had been tested only in film form, with no previous animatic rehearsal. [More than 50 ad pre-test results].

Database

The samples of commercials that were analyzed were all tested with Ameritest by Unilever as part of their normal business process. The composition of the four groups reflected roughly equal proportions of new versus established brands and was not dominated by the test results for any one or two brands. Indeed, the sample represented a wide range of brands in the home care product category (e.g., laundry detergent, softener) and in the personal care product category (e.g., hair care, face care, hand and body lotion).

Finally, it should be noted that this sample represents all the animatic-film test pairs for this set of brands over this time period and is not just an after-the-fact subset of ads for which improvements were found.

Research Metrics

The primary evaluative measure of breakthrough is Branded Attention, which is comprised of a measure of attention and a measure of branding. The primary diagnostic measure used in this study is the Flow of Attention™. These are all standard measures generated by the Ameritest system and additional detail can be obtained by visiting their website.

Key Findings

The statistical results of our study are shown in Table 1.Since these results are of a highly proprietary nature, all raw percents were converted to indices. All statistical test results reported below were based on raw percents.

Comparability of Animatic and Finished Film Scores

First, we compare the group of ads that were tested only in animatic form but not re-tested in film form (if they were in fact produced), with the group of ads that were tested only in film form, without animatic rehearsal first.

These results are consistent with previously published research, showing animatic scores nearly as strong as the scores of finished film. This provides additional evidence for the argument that animatic testing provides an efficient method for filtering advertising ideas, since the average performance of animatic and film ads are essentially equivalent. It should be noted, however, that we are only talking here about an average result. This is not the same thing as saying that all advertising concepts can be equally well-expressed in animatic form — an issue of concern to agency creatives striving to find fresh film techniques to express their advertising ideas..

Improvement from Rehearsal to Final Production

Next, we examine the scores of the fourteen animatic-film pairs The film version of the ads in this group scored +22% higher on Branded Attention than the animatics. Importantly, there was statistically significant improvement on all three evaluative measures of Attention, Brand Linkage, and Branded Attention. In short, there was a larger than expected increase in branded break-through performance for the same ad concepts as a function of rehearsal.

A somewhat surprising finding was that the average performance of the animatics in this second group was not substantially different from the average performance of the animatics in the first group of ads that were only tested in the animatic stage.. As noted above, all of the ads in group two were selected for final production and were re-tested in final film form. In contrast, while some of the ads in group one were produced, none were re-tested in final film form. Consequently, we might have expected higher average performance from the second group of animatics. This appears to contradict the primary function of animatic testing—which is to filter out the weaker ideas.

The explanation for this is that the animatics that were eventually re-tested in film were quite average performers which, for a variety reasons, went forward into production. In general, insights provided by the diagnostics would have suggested that these advertising ideas might be “diamonds in the rough.” That is, they might have untapped potential which could be released with further execution “polishing.” To allay management concerns about the original average scores, therefore, these ideas required additional testing after they were produced.

The Value of Rehearsal

Finally, when we compare the performance of final film for commercials that were tested first in animatic form versus commercials that skipped the rehearsal step and went straight to final film, we see the value of the rehearsal process.

Film ads that were rehearsed first in rough production scored +28% higher on Branded Attention than commercials that skipped the rehearsal step and went straight to final film. Again, we note statistically significant improvements in their Attention, Brand Linkage, and Branded Attention measures.

Three Case Studies

What follows are three case studies – taken from the sample of 14 ad pre-test results comprising the two ad groups described above – showing the benefits of “rough rehearsal” based on careful consideration of diagnostic insights from the Ameritest Flow of Attention® measure.

Case 1 – Suave Hair Care: Improving the Opening to Enhance the Attention Score

Opportunities for Improvement

The key challenge for this ad was softness on the Attention Score. Flow of Attention results showed that there was no initial hook to engage viewers at the outset of the commercial. To compound this problem further, the animatic had music but no voice-over for the first twelve of its entire thirty seconds.

RoughingIt_exhibit1

What Changes Were Made

Engaging close-ups of a child having fun on a merry-go-round were inserted at the outset of the commercial to enhance viewer interest and generate positive emotion. Additionally, the voice-over narrative started seven seconds earlier in the film version than in the animatic. Finally, the key discriminator – You don’t have to spend a lot to get a lot – was added to the copy to more clearly communicate the Suave point of difference.

Benefits of Rough Rehearsal

As shown in Figure 1, Flow of Attention results showed less of a drop in cognitive engagement at the outset of the final film ad compared to the animatic. In short, an opening that was slow in the rough version received added momentum in the film version thanks to the fun-filled, close-up images of the child on the merry-go-round.

The Attention Score increased 21%.

Case 2 – ThermaSilk Hair Care: Improving a Transition in the Middle to Enhance Brand Linkage

RoughingIt_exhibit1

Opportunities for Improvement

The key challenge for this ad was having a soft level of Brand Linkage, possibly because viewers weren’t processing the abruptly occurring brand slate that occurs in the middle of the commercial, as shown in the dip of the Flow of Attention in the upper panel of Figure 2. Nor do the “real hair” shots stand out in the latter part of the ad, possibly due to the abrupt transitions between the “thermal world” and the “normal world” shots, suggesting that viewers have a superficial understanding of the heat-activation benefits of the product.

What Changes Were Made

The transition introducing the brand slate in the middle of the ad was made less abrubt, moving smoothly from a shot of thermal-imagery hair to thermal-imagery product to normal product. To better communicate how the product works, the brand descriptor “Heat Activated” was added in the copy of the film version. Additionally, to clarify transitions between the “thermal” and “normal” worlds, more visual contrast and a more extended physical context was used (i.e., showing a complete head or profile rather than close-ups that were difficult to identify). Also, more screen time was devoted to the shots of “real” hair.

Benefits of Rough Rehearsal

Flow of Attention results showed that, compared to the rough version, viewers of the finished version were focusing on the brand slates in the middle of the commercial. The smoothed transition was easier to follow. Furthermore, viewers were not tuning out the real hair shots in the finished film version.

And Branded Attention increased 42%.

Case 3 – Degree AP/Deo: Improving the Ending to Enhance Brand Linkage

RoughingIt_exhibit3

Opportunities for Improvement

The key challenge for this ad was its soft Brand Linkage. Flow of Attention results showed that viewer engagement began to wane near the end of the commercial, when the branding slate was presented, and coincidentally, right after the story had been positively resolved. This commercial, in effect, had run out of story before running out of ad. There was also a focus of attention on the heat-seeking gadget rather than the brand. As a result, this ending brand slate was poorly recalled, as shown in Figure 3.

What Changes Were Made

The storyline was extended so that the story ended when the ad ended. In addition, inserting the shot of the android’s crushed eyeball re-energizes the momentum of the story and once again draws the viewer into the experience of the ad. Flow of Attention results showed that viewers were fully engaged right up to and during the ending brand slate.

Benefits of Rough Rehearsal

Flow of Attention results showed no downturn in cognitive engagement at the end of the film version compared to the animatic.

And once again, Brand Linkage increased 25%.

RoughingIt_exhibit1

Summary and Conclusion

From a pre-testing perspective, the results detailed above underscore the advertising effectiveness improvements that can be realized based on our recommendation of rough rehearsal. Also, from a financial perspective, we can calculate an approximate dollar value of this rough rehearsal process. These days a major advertiser such as Unilever will spend approximately five million dollars in media costs to air a typical television commercial. What our findings suggest is that Unilever can spend less than $50,000 on the rough production and research of an advertising idea in order to gain more than a $1 million improvement in the audience impact of the final commercial — a twenty-to-one return on investment!

Thanks to our rough rehearsal process, ads that would have been discarded based on an examination of “report card” measures alone have been salvaged and improved. Our results also support the general business practice of leveraging diagnostic insights at every step of the advertising development process. Finally, the improvements we see in commercial performance after rehearsal in rough form confirms that Creatives are indeed listening to the advertising research.

References

Kastenholz, J., Kerr, G., and Young, C. Focus and Fit: Keys to Well-Branded TV Commercials. Paper presented at the ARF Week of Workshops, Advertising Research Foundation, New York City, September 2003.

Pierce, P. Rough Versus Finished Commercials in Research. In J. P. Jones (Ed.), How Advertising Works: The Role of Research. Thousand Oaks, CA.: Sage Publications, Inc., 1998.

Senge, P. The Fifth Discipline: The Art and Practice of the Learning Organization. New York: Doubleday, 1990.

Young, C. Capturing the Flow of Emotion in Television Commercials: A New Approach. Journal of Advertising Research, 44 (1), March 2004, in press.