An Innovative Approach in a Growing Field: How We Moved the Dial on Social Media Evaluation

Theoretical Background and research questions/hypothesis:  Social media has added a new dimension to communication. It is increasingly being used in nearly every aspect of life. There is enthusiasm and interest in using social media as a tool for public health communications, but there is little understanding of the connection between online engagement and behavior change.  Many public health organizations use social media to amplify messages from traditional media sources and as a strategy to engage with target audiences. There is modest evidence that interventions using online social networks are effective at changing behavior; however, this field of research is still young. Currently, evaluating behavior change interventions on social media is challenging. Frameworks for this type of evaluation are still evolving, and many types of social media are designed without an evaluation framework (blogs, social networking sites, etc.) Most often, the metrics that programs collect include traffic on websites, influence and reach, as well as user interactions and engagement (likes, retweets, shares etc.). The impact of social media on health and behavior is harder to measure. More information is needed to understand how social media impacts offline health behaviors. It is important to identify the most effective strategies to reach social media users and what types of messages cut through the online clutter.

Methods and Results (informing the conceptual analysis): We developed an evaluation tool to identify and categorize social media posts by topic and behavioral construct. We collected and tracked data from our three social media channels over six months.  Data were categorized into five topics: 1) blogs, feature stories, and publications; 2) trending topics; 3) incidents; 4) emergency operation center activations; 5) facts, information, and news and included social media measures (e.g., engagement rate, impressions, etc.), graphical content, and time of day. Each post was also coded for knowledge, attitudes, and behaviors; theoretical constructs; Bloom's Taxonomy domains; and emotion. Confidence intervals were established around the average engagement rate for all five topics and each post was assigned a low, medium or high rating. For the posts coded by construct, a random number generator table was used to check for inter-rater reliability. Two separate reliability checks were conducted.  In terms of knowledge, attitudes and behaviors, the majority of Twitter posts were coded ‘knowledge’ (60%); 22% were coded as ‘behavior’. Seventy-three percent of Facebook content was coded as ‘knowledge’. The majority of both Twitter and Facebook posts fell in the ‘Apply’ and ‘Understand’ domains of Bloom’s Taxonomy.  Ten percent of Facebook content and 20% of Twitter posts contained information about the risk of disease or public health emergency. 

Conclusions: Further research is needed to identify how best to maximize audience retention and engagement; whether behavior change can be sustained long-term as a result of exposure to content on social media; and to determine how to leverage social media networks for mass dissemination of public health messages.

Implications for research and/or practice: The next steps are to develop, implement, and evaluate a public health intervention on social media to validate and build on the results of this research.