Interface Considerations Regarding Authorship of Computable Ph Case Definitions

In this manuscript we report an evaluation of the reliability of biosurveillance case definitions creation by multiple clinicians using the Health Archetype Language (HAL-42) and user interface. The HAL-42 language allows real time epidemiological inquiry using automatically derived clinical encodings with any health Ontology. This evaluation used SNOMED CT as the underlying Ontology. The inquiries were performed on a population of 17,731 patients whose 50,000 clinical records have all been fully encoded in SNOMED CT. Four subject matter experts (SMEs) were asked independently to encode and run ten rules / studies. The inter-rater agreement was 74.8% (p = 0.6526) with a Kappa statistic of 0.49217 (p = 0.5722). The ten rules were divided into three easy rules, four moderate and three complex rules. There was no significant difference in the SME’s agreement when representing easy and complex rules (p = 0.6243). We conclude that although the usability of the HAL-42 language is usable enough to achieve reasonable inter-rater reliability, some training will be necessary to reach high levels of reliability for ad hoc queries. We also conclude that SMEs are just as competent to perform complex queries as easy queries of ontologically indexed clinical data.