Using Non-Probability Web Surveys to Measure Sexual Behaviours and Attitudes in the British General Population

Background: There is increasing interest is using non-probability web surveys from access panels for collecting epidemiological data, as they can provide a relatively cheap and quick alternative to traditional surveys. However, studies in other countries have raised concerns about their lack of representativeness. This is the first study in Britain to compare results from web panel surveys from a number of research agencies with each other. Results are also compared with the third British National Survey of Sexual Attitudes & Lifestyles (Natsal-3), a probability sample interview survey.

Methods: Natsal-3 questions were included on four non-probability web panel surveys administered by three commercial research agencies. Two of the surveys used ‘traditional’ quota controls of age and sex, and two used ‘modified’ quotas with additional controls on variables related to key estimates. Results were compared with external benchmarks for participant characteristics and with Natsal-3 data for sexual behaviours and attitudes.

Results: Compared with external data, results for all four web surveys were less representative of the general population than Natsal-3. Moreover, for all four web surveys, around two-thirds of the key estimates of sexual behaviours were significantly different from Natsal-3. There was no one web survey that consistently performed better than any other. The modified quota web surveys slightly improved results for males, but not for females.

Conclusions: Consistent with results found elsewhere, web surveys in Britain using volunteer panels are unlikely to provide representative estimates for the general population. The use of more sophisticated quotas may lead to some improvement, but many estimates are still likely to differ. Surveys of volunteer web panels may be useful in some circumstances, but not if accurate prevalence estimates are a key objective of the study.