Message Testing and Evaluation: When a National Media Campaign Turns Local

Background:  CDC's Communities Putting Prevention to Work initiative funded 61 programs in 50 communities to reduce chronic disease related to obesity and tobacco. Through evidence-based strategies, CPPW communities worked to improve access to healthy food, increase opportunities for physical activity, and reduce tobacco use and exposure to secondhand smoke. These efforts are expected to produce broad, sustainable health outcomes through environmental-change strategies.

Program background:  The National Prevention Media Initiative was intended to support community efforts by creating a national dialog around unhealthy behavior norms, the influence of the nutrition and physical activity environments on people’s ability to practice healthy behaviors, and how people in communities can work together to effect changes that make healthy living easier.

Evaluation Methods and Results:  Throughout the various iterations of creative development we conducted concept and message testing. This presentation will focus on the last round of testing – after communities were allowed to decide what advertisements would be placed locally with CDC funds, rather than CDC fielding a nationally branded campaign. Plans for evaluating what became 61 different media initiatives, and the timeline for that evaluation, will also be discussed. Of the 50 CPPW communities, 39 communities worked on obesity prevention, focusing on a variety of topics and using a variety of strategies and tactics. The 12 ads chosen for testing included 4 print ads, 3 radio ads, and 5 TV ads. Topics covered in the 12 ads included physical activity, physical education in schools, sugar-sweetened beverages, and obesity trends. Through an online panel, the ads were tested for acceptance, including measures on whether the ads were confusing, if they were made for “a person like me,” and what grade (A, B, C, D, or F) the ad should receive. This testing was to help determine if ads created by one community would be accepted if used by another community. The online panel sample size was 800. Rotating the ads allowed approximately 400 people to review any given ad.

Conclusions:  Overall the ads tested well. There were slight variations by region for some of the ads. For instance, the “Eat Sugar” TV ad from New York City and the “American Children” TV ad from Hawaii tested better in the Northeast than in the Midwest. Also, the “Mom Driving” TV ad was rated as confusing by more people in the South than in the Northeast or Midwest.

Implications for research and/or practice:  While we might expect media can have a strong and positive impact on obesity prevention, we do not yet have the accumulated evidence that this is the case, as we do for anti-tobacco media. Thus, our campaign evaluation efforts will focus on the 39 CPPW obesity communities, each with separate local media initiatives. In lieu of a formal baseline survey, we will be analyzing geocoded data from the nationally representative fall 2011 Porter Novelli HealthStyles survey. We also are planning to analyze other existing sources of national and local CPPW survey data and to field a telephone survey in the 50 CPPW communities to obtain additional metrics around attitudes, beliefs, and aided awareness of advertising.