Planning a Multisite Evaluation of Inter-Organizational Collaboration and Service Integration in STD Programs

Background: In order to better integrate STD care, we needed quantifiable process and outcome measures to evaluate the relationship between collaborative efforts, services and data sharing.  

Objectives: To develop a multi-site evaluation (MSE) plan for the three-year demonstration project to scale up Program Collaboration and Service Integration (PCSI).

Methods: An evaluation workgroup was convened from April 2011-June 2011, with representatives from each demonstration project area and CDC program and teams. The group identified various approaches to the evaluation, discussing core measures, evaluation questions and framework. The evaluation team identified appropriate theory to ground the evaluation and facilitated the planning process, involving the workgroup in each phase of planning.

Results: The final evaluation plan includes process measures to be evaluated in year one, with inter-organizational collaboration being the main measure. Years two and three focus on process, outcome and some impact measures. Referencing the CDC Framework, core activities were identified in a matrix. A global logic model was developed to depict common activities and measures. Evaluation questions were developed according to two overall measures: program collaboration and service integration. A collaboration assessment rubric was adapted from the Communities of Practice model.  The workgroup concurred that qualitative and quantitative approaches to data collection would be used.

Conclusions: A MSE plan was developed by the workgroup using the participatory principle. Components of the evaluation include: evaluation framework, program improvement plan, logic model, evaluation matrix, sample indicators and collaboration assessment rubric. This evaluation will provide insight on scaling up integrated care and affect future STD care.

Implications for Programs, Policy, and Research: The process undertaken in the planning of this evaluation has programmatic implications regarding finding measures among programs with varying interventions, quantifying collaboration as a measure, developing strategies to ensure feasibility of evaluation plan components and obtaining buy-in from stakeholders regarding evaluation approach.