Consultation response
Participation and prevalence research
From December 2020 to February 2021 we consulted on the proposals to change how we collect adult gambling participation and problem gambling prevalence statistics.
Contents
- Executive summary - participation and prevalence research
- Introduction - participation and prevalence research
- Summary of responses - participation and prevalence research
- - Introduction to summary of responses
- - Proposal 1: Single survey for Great Britain
- - Proposal 2: Consolidation of current surveys
- - Proposal 3: Participation questions
- - Proposal 4: Frequency and turnaround time
- - Proposal 5: Explore more future proof methods
- - Proposal 6: Survey pilot
- Annex A: List of consultation responses
Proposal 6: Survey pilot
Issue
Changing the survey method could result in changes to the data.
Proposal
- to pilot questions using a potential new methodology in 2021 so that we can compare the results of the pilot with the telephone and online surveys that take place over a similar time frame, and with the most recent Health Survey data (2018)
- to take steps to ensure that the survey does not encourage an over-representation of gamblers, by taking care in the way the survey is branded and introduced to participants
- to analyse and report on comparability of trend data
- subject to satisfactory pilot study data, to begin our new survey methodology in 2022.
Consultation question boxes:
- Q15. Do you agree with this proposal?
- Q16. Would a break in the time series impact your work?
Respondents’ views
Most respondents agreed with Proposal 6; to pilot questions using a new methodology in 2021. 40 out of 61 respondents agreed with the proposal, of which 14 strongly agreed and 26 agreed. Five respondents who strongly agreed with the proposal were from academia. Just seven respondents disagreed with the proposal, with more strongly disagreeing (5) than disagreeing (2). A total of 14 respondents neither agreed nor disagreed with the proposal, with 10 of these respondents being members of the public.
Those who agreed thought that a pilot was a sensible approach to take, with one respondent saying that the Commission should take advice on the size and the scale of the pilot required to be able to robustly compare results with previous methods. The aim of the pilot should be to give users trust and confidence in the accuracy of the data going forward.
There was some concern about the timing of the pilot given the impact of Covid-19 on gambling behaviours. Respondents thought that results from a 2021 pilot which are compared to results collected pre-Covid (e.g. the Health Survey England 2018) need to be used with caution as differences may be due to changes in the gambling environment as a result of Covid rather than methodological differences. Similarly, comparisons to the 2016 Health Surveys should also be made with caution, given that the fieldwork took place five years ago and the gambling environment has changed since then. Respondents seemed to think that the solution should be comparing the results of the pilot study to a variety of other data sources rather than just one. This should include Gamble Aware's methodology review (opens in a new tab) or the 2021 Health Survey, although it is acknowledged this would cause a delay for the Commission’s methodology review.
One of the respondents who disagreed with the pilot approach thought that rather than piloting a new survey approach we should be overhauling the whole research programme to include the use of administrative data for survey validation.
Finally, a request for the outcome of the pilot to be published was made by an operator.
When asked if a break in the time series of data would impact their work, most respondents were of the mind that it would not. As one respondent said, they have managed with the uncertainty of prevalence estimates for some time (due to mode effects) and therefore if a break in time series means that the data quality and reliability is improved it is worth doing. Another said that they do not use data from the Commission’s quarterly telephone survey because the full dataset is not currently publicly available so for them a break in time series would not be an issue.
One respondent who thought the break in time series would impact their work was concerned that it would impact their work in relation to safer gambling and could lead to an increase in the misuse of data.
Figure 6: Would a break in the time series impact your work?
Base: 59 responses
Data from pie chart
| Yes | No | Undecided | |
|---|---|---|---|
| Number of responses | 7 | 36 | 16 |
Our position
Given the level of agreement with our proposals in this consultation document we plan to pilot a new approach in 2021/22.
We will run a procurement exercise to find a suitable research partner to lead the pilot, with the pilot having two aims:
- to test current participation and prevalence questions using a new methodology against data from a range of existing surveys
- to test new questions aimed at measuring participation using a refreshed and updated list of gambling activities.
We will consider options for the best way for the pilot to achieve the above aims including running a two-stage pilot or a split sample within one pilot survey.
We will take on board findings from the GambleAware research methodology project.
We will publish the outcome of the pilot and our plan for a future survey methodology.
Proposal 5: Explore more future proof methods
Last updated: 23 June 2021
Show updates to this content
No changes to show.