This box is not visible in the printed version.
This pilot involved testing a new push-to-web methodology to measure gambling participation, gambling harms, experience of problem gambling and other related topics.
Published: 24 May 2022
Last updated: 10 July 2023
This version was printed or saved on: 28 April 2024
Online version: https://www.gamblingcommission.gov.uk/about-us/guide/participation-and-prevalence-pilot-methodology-review-report
Overview: ## Pilot methodology review report
In December 2020, the Gambling Commission launched a consultation to gather views on proposals to develop a single, high-quality methodology to measure gambling participation and prevalence of problem gambling.
The aim was to have a more efficient, cost effective data source providing robust and timely insight and the flexibility to swiftly provide information on emerging trends relating to a range of gambling behaviours.
The outcome of the consultation was published in June 2021. The Commission launched a competitive tendering process for a contract to pilot a new data collection methodology between 2021 and 2022. The contract was awarded to NatCen Social Research (NatCen), working with the University of Glasgow and Brydon Purdon Social Research to conduct initial piloting to assess a new methodology for this study.
This pilot involved testing a new push-to-web methodology, using random probability sampling, to measure gambling participation, gambling harms, experience of problem gambling and a range of other related topics among adults aged 16 and over living in Great Britain.
The pilot is the first stage in a range of ongoing development work. The next stage is to roll out a further period of data collection commencing in summer 2022, badged as experimental statistics to allow further testing and development. Subject to the success of the experimental phase the survey will move, in July 2023, to continuous official statistics data collection.
This pilot methodology report outlines the pilot survey push-to-web methodology and provides detail on the response rates achieved. The report also evaluates the pilot survey data collection phase and makes recommendations for the next phases.
In order to generate generalisable estimates of gambling participation and experience of gambling problems among the British population, it is important to use random probability sampling methods.
Non-probability methods, such as online panels, tend to have elevated rates of problematic gambling compared with probability methods, with estimates of problem gambling ranging between 2 to 3 percent for the population, compared with around 1 percent from probability methods.
Many national surveys, especially in response to the Covid-19 pandemic, are moving to a push-to-web approach which uses offline contact methods to encourage people to go online and complete a questionnaire.
To minimise non-coverage and selection bias, online surveys are often paired with an alternative completion mode (for example, as for this study, a shorter, more focused, postal questionnaire which enables less technologically literate people and/or those without internet access to take part).
Push-to-web is a cost-effective method of collecting data from a wide number of people. It also allows increased numbers of people to be interviewed at relatively lower cost, something that is important for the analysis of gambling harms. However, this is a different approach to how previous studies of gambling have collected data; the Health Survey for England (HSE) and British Gambling Prevalence Surveys have relied on interviewers personally visiting addresses to conduct interviews and the Gambling Commission’s quarterly survey collects information via telephone survey using Random Digit Dialling (RDD).
The focus of this pilot was to:
In total, 3,755 addresses were sent an invitation to take part in the survey and up to two adults aged 16 and older from each were invited to take part. The adjusted address-level response rate, i.e. the proportion of eligible addresses where a questionnaire was completed by at least one adult aged 16 or older in eligible addresses, was 21 percent.
Results in the report are based on 1,078 adults who fully completed the questionnaire. In total, 44 percent of the responding unweighted sample were men and 56 percent were women. Overall, 57 percent (619) completed the survey online and 43 percent (459) completed a postal questionnaire.
The evaluation of the methodology change focused primarily on gambling participation, prevalence of problem gambling and at-risk Problem Gambling Severity Index (PGSI) scores.
The initial stage to evaluating the pilot survey approach was to test the suitability of the push-to-web methodology for collecting gambling participation and the prevalence of problem gambling data in the future as well as to, assess how the results compared to the estimates generated from the self-completion element of the Health Survey for England (HSE) 2018 and trend-adjusted HSE figures.
The pilot survey produced higher prevalence estimates - both for gambling activities in the previous 12 months and for low or moderate risk or problem gambling - in comparison with both HSE 2018 and the trend-adjusted HSE figures. For example, 54 percent of HSE 2018 participants reported having gambled in the previous 12 months compared with 63 percent of those from the pilot survey.
Within the pilot survey, 1.3 percent of participants had a PGSI score of 8+ (problem gambling) compared with 0.4 percent in HSE 2018. The respective figures for moderate risk gambling was 2.3 percent and 0.8 percent and for low risk gambling 7.8 percent and 2.7 percent. The results from the pilot survey should not be used as new estimates of gambling participation or the prevalence of problem gambling.
There was little difference between the surveys in relation to the prevalence estimates produced for older people. However, the pilot survey produced higher estimates for younger people, particularly women, notably in relation to online gambling rates and at risk and problem gambling prevalence.
The higher prevalence estimates produced by the pilot survey were not unexpected. The recent review by Sturgis and Kuha (opens in a new tab) concluded that online surveys tend to over-estimate gambling participation and it is also possible that health surveys may under-estimate prevalence.
Our analysis highlighted two potential causes of differences:
Comparing participants from the two surveys with similar gambling activity profiles, pilot survey participants were more likely to score one or more on the PGSI than their counterparts in HSE 2018. This needs further investigation but may be because people provide more honest answers when reporting behaviours online than when filling in self-completion questionnaires when an interviewer or other family members are present.
The pilot was successful in attracting participants and exceeded response rate expectations.
Estimates of gambling participation and problem gambling were somewhat higher than those based on the Health Survey for England (HSE) but were lower than those typically generated by online panel surveys and thus broadly commensurate with expectations at this stage. We have also identified two potential explanations for this, outlined in the Assessment of pilot survey results section, which lend themselves to further development work.
Based on these findings, we recommend that the push-to-web methodology is suitable for roll out to the experimental statistics stage, subject to the recommendations detailed in chapter 7 of the pilot methodology report.
The key recommendations are as follows: