Using Online Message Testing to Evaluate TV Ads, Select Effective Messaging and Defend Your Campaign

Background:  The Centers for Disease Control and Prevention recommends using mass media campaigns to change attitudes about tobacco use and encourage people to quit. Although proven effective, mass media campaigns – particularly television – can be expensive and controversial. Online message surveys are an economical and effective way to assess television ads for important purposes, including to decide which ads are most effective with your target audience, to evaluate your agency’s work and to defend against criticisms of ads.

Program background:  For a decade ClearWay Minnesota has been creating mass media campaigns to educate people about the dangers of tobacco use and secondhand smoke and to motivate people to quit. These campaigns have included TV, radio, Internet, billboard and print advertising. With its relatively low cost and short turn-around compared to longer-term evaluations, online message testing has become an important component of running a successful mass media program for ClearWay Minnesota.   

Evaluation Methods and Results:  ClearWay Minnesota has conducted several online message testing surveys in recent years. Participants included both tobacco users and non-users belonging to established, online research panels who live in Minnesota or the adjoining states. Action Marketing Research developed the questionnaires in collaboration with ClearWay Minnesota and fielded the surveys.  Participants were randomly assigned to see a series of videos selected for testing. Respondents had to verify that they had seen and heard the spots to continue on to the online survey questions. Within each survey, iterative weighting was applied to the data.  There were several key findings from these online surveys that were used to strategically inform our work.  First, in one of the surveys ClearWay Minnesota tested its existing creative, comparing them with TV ads from another state and from the CDC’s Tips campaign. The respondents found the CDC ads to be the most motivating and the most helpful of the three campaigns. ClearWay Minnesota determined this campaign would be most effective and ran the CDC’s ads.  Second, ClearWay Minnesota launched a new TV campaign in 2012. Testing was used to assess the strength of the ads post-production and evaluate the ad agency’s work. All three spots successfully communicated key messages and scored high on other factors.  Third, recently one of ClearWay Minnesota’s ads was publically criticized. Using the online message testing results ClearWay Minnesota effectively responded by articulating how the ads were interpreted by the public. 

Conclusions:  Online message surveys are an economical and effective way to support testing television ads for several important purposes including: to decide which ads are most effective in communicating with your target audience, to evaluate your agency’s work and to defend against criticisms.

Implications for research and/or practice:  Online message surveys will improve program effectiveness and protect investment in marketing campaigns.