Beyond An Increase in Knowledge: Assessing the Impact of Training

Objective: To describe an evaluation conducted to assess the impact of changes in practice due to training events

Methods: The Utah Diabetes Telehealth Series is a monthly distance-learning program designed to update health care professionals on the latest information in diabetes treatment. Approximately 125 health professionals attend each broadcast. Pre- and post-tests are used to measure increases in knowledge, but little has been known about the impact of the broadcasts on practice change. Five broadcasts, which focused on women’s care, were selected for a pilot project to determine whether the broadcasts had an impact beyond a simple increase in knowledge.  In July, 2009, an online survey was sent to attendees for each broadcast asking the clinical setting, number of women who may have been affected by a practice change, and any type of change made. The time between each broadcast and the distribution of the July survey ranged between three and thirteen months.

Results: A total of 113 respondents participated in the survey. The number of women seen in practices of the broadcast attendees exceeded 8,000. Over half (54.5%) of attendees for the health literacy broadcast reported they made a practice change.  About one of five (20%) attendees made a practice change after viewing heart health, retinopathy, and polycystic ovarian syndrome broadcasts. One-seventh (14.3%) made a change after the gestational diabetes broadcast.

Conclusion: It is possible to assess the value of training beyond an increase in knowledge. Innovative tools such as online surveys provide a useful mechanism for assessment of practice change over time.