Development of An Immunization Forecasting System: From Paper to Prediction

Background:  Immunization forecasting systems are automated software algorithms that assess patient immunization status and recommend the dates and vaccines that should be administered next.   Forecasting applications are integral to the management of childhood immunizations, but are costly to purchase and integrate into electronic medical record (EMR) systems.  For these reasons, Texas Children’s Hospital (TCH) developed a forecasting system linked to its EMR for use in its pediatric facilities.

Setting:  The forecasting system was developed collaboratively by a physician, nurse, software engineer, and immunization experts.  Team members were located in Texas and Oregon and met in person and virtually.

Population:  N/A

Project Description:  To improve the accuracy of immunization recommendations within TCH’s EMR, a forecasting system was developed.  A two-step process involving manual and electronic review was implemented to validate the accuracy of the forecasting system.  Manual validation included the review of approximately 2,000 immunization records that were randomly selected and varied by age. For the electronic validation, TCH obtained test cases from a state IIS and a third-party forecasting system and compared the immunization predictions obtained from each to the desired TCH forecasting system results, using an interactive web-based application developed specifically for this purpose. This program allowed team members to manage test cases, identify errors and solutions, and facilitated overall communication.  This presentation will include a demonstration of the interactive web-based testing program.

Results/Lessons Learned:  Some immunization recommendations were difficult to operationalize including the live virus vaccine minimum intervals and variations in the Td/DT/Tdap catch-up schedule. Although testing and verifying results can be tedious, a rigorous process is essential to the development of a reliable forecasting system. An interactive web based program is critical for testing and validating forecasters, especially when working remotely. The result of the collaboration between technical and clinical experts was a forecasting system that produces accurate and reliable immunization predictions.