37173 Evaluating Altmetric™@CDC: Assessing Benefits of Using Real-Time Scholarly Content Data to Inform Scientific Work

Adela Santana, MPH, MSEd, CHES1, Deidre Mallett, Masters in Library Science, Masters in Health Sciences2, Nancy Gathany, PhD, MEd3, Jessie Hood, ScD, MPH4, Nancy Habarta, MPH5 and Conne Ward-Cameron, BA4, 1Division of Public Health Information, Centers for Disease Control and Prevention (CDC), Atlanta, GA, 2Library Science and Services Branch, Division of Public Health Information Dissemination, Centers for Disease Control and Prevention (CDC), Atlanta, GA, 3Centers For Disease Control and Prevention, 4Centers for Disease Control and Prevention (CDC), 5Centers for Disease Control and Prevention

Background:  Thousands of conversations about scholarly content happen through various media channels every day. For public health institutions that produce widely-circulated scholarly content, understanding and interpreting the attention this content receives online in real-time can potentially help programs to see who is interested in their work and whether it is being accurately represented and interpreted. In February of 2016, the Centers for Disease Control and Prevention’s (CDC’s), Stephen B. Thacker CDC Library, introduced Altmetric™@CDC, a digital tool that allows users to see the attention that their online publications are receiving in real-time from social and mainstream media and what people are saying about their work. Developed by Altmetric, Inc., the online web-based platform, Altmetric™@CDC aggregates all of the information it can find about a scholarly article. Looking at both the quantity and the quality of attention being paid to an article, the information is weighted and turned into a score, which then appears in a colored circle referred to as the “Altmetric™ donut.” In addition to viewing the score, staff can view the original sources of citation, sign up for alerts, and access bibliographic details. To ensure effective use of Altmetric™@CDC during its initial implementation, CDC Library staff are aiming to provide training and technical assistance to a critical mass of CDC staff.

Program background:  Altmetric™@CDC has the potential to provide several benefits to CDC, its programs, and staff. The data available through Altmetric™ reveals a deeper context for understanding and interpreting how public health content is received, and how audiences engage and respond to the information. By obtaining a greater understanding of the societal engagement, influence and broader impacts of research, CDC staff may be able to use Altmetric™ data to inform program health communication and marketing strategies. For example, staff at CDC are currently exploring the possibility of using Altmetric™ data to strategically craft health communication messages, select channels for dissemination and target audiences. 

Evaluation Methods and Results:  As a new program to CDC, it is critical to incorporate evaluation of Altmetric™@CDC from the start of the program to support successful implementation and assess any theorized benefits. Ongoing collection of evaluation data, through semi-structured interviews, focus groups, and an online customer satisfaction survey, will provide information on CDC staff’s level of awareness, use, and satisfaction of the program. In addition, evaluation data will demonstrate any potential benefits to CDC staff gained from using Altmetric™@CDC. 

Conclusions:  Evaluation data on Altmetric™@CDC on awareness, use and satisfaction is continuously being fed back to program leadership to inform programmatic decisions. For instance, examples of use among CDC staff will inform development of training and technical assistance. 

Implications for research and/or practice:  The program implementation and evaluation of Altmetric™@CDC is an introduction of a digital tool with the potential to innovatively support the scientific work of CDC. Experiences from implementation and early evaluation of this tool will provide lessons learned to other public health institutions that also produce widely-circulated scholarly content and want to consider offering a similar tool that provides real-time data on the attention this content receives through social and mainstream media.