Evaluating IIS Best Practice Operational Guidelines: Emerging Trends and Challenges

Background: In 2005, the American Immunization Registry Association (AIRA) in partnership with the Centers for Disease Control and Prevention (CDC) formed the Modeling of Immunization Registry Operations Workgroup (MIROW) to develop a best practices operational improvement guidebook for immunization information systems (IIS).

Objectives: Participants will learn about a) status of and future plans for the MIROW efforts and b) results of evaluating IIS community awareness, acceptance, utilization of and satisfaction with MIROW guidelines.

Methods: Various evaluation approaches have been used:

• Questions regarding utilization of MIROW guidelines were included in the Immunization Information Systems Annual Report (IISAR).
• A web-enabled survey has been conducted to evaluate best practice guidelines for one of the MIROW topics – management of patient active/inactive immunization status.
• After every workshop, a questionnaire was completed by participating experts to gather feedback on various aspects of the best practice development efforts.

Additionally, informal feedback gathering approaches were utilized, e.g., case studies of guidelines implementation in state IIS, ad-hoc meetings at immunization conferences, and external reviews of guidelines.

Results: Evaluation results indicate a growing acceptance and implementation of MIROW guidelines.

• IISAR data indicate that 46% of grantees were using MIROW guides in 2007 and 76% - in 2008.
• Eighteen of the 19 IIS participants of the web-enabled survey who were familiar with the guidelines on management of patient active/inactive status found them to be at least somewhat useful in their work.
• A numerical score for the average satisfaction with recommendations framework among participants of the last four workshops is 3.56 (on a scale of 1 to 4).

Conclusions: While IISAR results indicate awareness and recognition of the MIROW best practice operational guidelines in the IIS community, additional studies are needed to evaluate specifics of guidelines’ acceptance and utilization.