Assessing Variability Among IIS Vaccine Forecasting Algorithms

Background: The Advisory Committee on Immunization Practices (ACIP) issues detailed recommendations for vaccination of children, adolescents, and adults. These recommendations serve as an authoritative reference for immunization providers on what vaccinations(s) should be administered to individuals now and in the future. Immunization Information Systems (IIS) develop clinical decision support tools (forecasting algorithms) that automatically determine needed routine immunizations following current ACIP recommendations, given an individual's immunization history to date. IIS forecasting algorithms have been developed independently and are in use by immunization program grantees and their providers.

Objectives: Measure variability of IIS-forecasted administration dates for specific vaccine doses compared to ACIP recommendations.

Methods: Using 2008 annual report data, CDC identified 44 IIS grantees as having an IIS forecasting algorithm tool.   CDC developed a web-based survey consisting of two questions pertaining to vaccine shortages and possible impacts, and nine vaccine forecasting test cases. After piloting to select IIS grantees, the survey was distributed to all 44 grantees.  Variability was measured by the number of days between IIS-forecasted vaccine due dates and dates forecasted by an ACIP subject matter expert (SME).

Results: Thirty-six (82%) of 44 IIS responded to the survey. Results indicated that interpretation of ACIP recommendations varied among IIS vaccine forecasting tools. Nineteen (53%) IIS forecasted vaccine administrative dates that would have rendered administered doses invalid, according to an ACIP SME. Invalid doses require repeat vaccinations.

Conclusions: Ambiguity in interpretations of ACIP recommendations leads to administration of invalid vaccine doses and extra-immunizations. IIS and vaccination providers would benefit from an unambiguous and standardized, computer-readable interpretation of ACIP recommendations for development and maintenance of vaccine forecasting algorithms. Perceived benefits may include (1) more accurate forecasting and reminder/recall, (2) decreased development and maintenance costs, (3) reduction in over-immunization and, therefore, decreased vaccine and administrative costs and (4) reduced risk to vaccine recipients.