Comparing Television Ads - Using Key Indicators to Evaluate Effectiveness of Ads

Background:  Mass-reach health communication interventions are one of CDC’s Best Practices for Comprehensive Tobacco Control Programs, and television ads are seen as a required vehicle. Media evaluations play an important role in assessing ad effectiveness, but can take significant time to complete a sophisticated analysis of the data. This potentially limits the timeliness of information for marketers to use in making campaign decisions. 

Program background:  ClearWay Minnesota uses mass media-campaigns to change social norms around tobacco and to promote its cessation services, QUITPLAN Services. From 2010 to 2013, ClearWay Minnesota embarked on a longitudinal and cross-sectional media evaluation, which included four waves of data collection. The purpose was to assess the effectiveness of its advertising programs and make short-term decisions about strategic direction. In order to easily evaluate and compare individual ads, a summary sheet was created comparing the ads by ad awareness, ad impact, ad awareness relationship to beliefs and, for the last wave, perceived effectiveness of the ad.

Evaluation Methods and Results:  The media evaluation collected data at four time points - October 2010, May 2011, May 2012 and May 2013. Individual ads were tested using the following measures: Ad awareness measures a participant’s recall of the ad using five questions: a “visual” awareness question, two “event” questions and two “theme” questions. Ad impact measures how influential the ad is on participants. It was evaluated with three questions that measured “talk about the ad”, “ad memorability” and “whether the ad provided new information or perspective”.   The relationship between ad awareness and level of agreement to specific ad belief was measured by asking about a specific belief from the ad. During the fourth wave, additional questions were added to measure the perceived effectiveness of the ad. This assessed how well the ad resonated with participants. The measurement consisted of four attributes: “stop and think,” “grabbed my attention,” “believability,” and “want to quit.” (Davis, et al., 2011) Using the data from the media evaluation on these key measurements, ClearWay Minnesota created a summary sheet for the ads that rated them on their effectiveness. This tool was used by the marketing department to assess and compare ads in determining future messaging strategies. An example of the summary sheet will be shared during the presentation.    

Conclusions:  While full analysis from an extensive media evaluation is important, decision-making for marketing often requires quick access to data on the effectiveness of ads. Using consistent and reliable measurements of ad effectiveness helps marketers make informed decisions on managing media campaigns. Creating a tool summarizing the key measurements of the ads makes it easy to compare and evaluate individual ads. 

Implications for research and/or practice:  Practitioners and evaluators need to find ways to quickly assess an ad’s effectiveness and impact after the implementation of a media evaluation, prior to the full analysis. Having a consistent methodology and criteria for measuring ad effectiveness and creating a quick summary tool to provide a snapshot of key variables will help marketers compare ads and manage media campaigns.