Cookies on the Gambling Commission website

The Gambling Commission website uses cookies to make the site work better for you. Some of these cookies are essential to how the site functions and others are optional. Optional cookies help us remember your settings, measure your use of the site and personalise how we communicate with you. Any data collected is anonymised and we do not set optional cookies unless you consent.

Set cookie preferences

You've accepted all cookies. You can change your cookie settings at any time.

Skip to main content
  1. Home
  2. About us
  3. Participation and prevalence research

Consultation response

Participation and prevalence research

From December 2020 to February 2021 we consulted on the proposals to change how we collect adult gambling participation and problem gambling prevalence statistics.

Published
23 June 2021

Last updated
23 June 2021 - Changes

Document actions
Print or save

Contents

Proposal 5: Explore more future proof methods

Issue

Traditional research methods, already in decline are now under greater threat due to Covid-19.

Proposal

To explore more ‘future proof’ methodologies for ongoing measurement which will be able to withstand threats posed to more ‘traditional’ research approaches. These methods include online, 'push to web' and mixed-mode surveys.

Consultation question

  • Q14. Do you support the proposal to use an alternative, non-interviewer-led methodology?

Respondents’ views

Over half (34 out of 62) of respondents supported the proposal to use an alternative, non-interviewer-led methodology in future surveys. Whilst 8 respondents did not support the proposal, 20 respondents were undecided on whether to support it or not.

Those who were in favour of a non-interviewer-led approach thought that a push-to-web approach offered the most viable alternative to face to face interviewing, and the most logical approach for the future especially given the Covid-19 pandemic. For a standalone bespoke gambling survey, respondents also thought a push to web survey would be the most cost effective solution. An online approach would also meet the Government’s digital transformation strategy.

Some respondents thought a non-interviewer-led approach would also reduce the impact of social desirability bias which might be an issue with some methodologies (e.g. telephone survey) given the sensitive nature of gambling questions.

There were some notes of caution however, the first about the importance of a random probability sampling approach rather than relying on self-selected online panels. Some were also cautious about an online only approach and stressed the importance of a mixed mode methodology to ensure representation across the population, including the idea of boosting response rates amongst some population groups, or even recruiting sub panels to take part in rapid response research.

Respondents provided some good examples of large scale surveys that had already made a similar methodology switch or used a mixed mode approach such as the Active Lives Survey, the OPSS Consumer Tracker Survey which consists of 10,000 online and 500 telephone surveys with those with low or no internet access, the Legal Needs of Individuals Study (LSB/Law Society) with 30,000 online responses and an offline calibration to determine weighting and general population surveys in Massachusetts which have used address based sampling and mixed mode methods of completion since 2013. Reference was also made to one research agency’s ‘Knowledge Panel’ which is a random probability online panel with over 10,000 panellists.

An acknowledgement that all research methodologies have pros and cons was made in the consultation response and for this reason a pilot study and the opportunity to run surveys using different methodologies side by side seemed like a good idea. A respondent also repeated their suggestion that the Commission should make more use of administrative data to validate survey responses, and another thought we should be considering incorporating alternative, less traditional methodologies such as netnography (Twitter scraping), ethnography and videography into our research programme.

Those who were not in favour of the proposal considered face to face interviewer led research to be superior to other methodologies and were not as confident in an online approach. There was also a concern about the loss of comparability of data over time due to a methodological change.

Whilst Covid-19 is currently prohibiting face to face interviewing, some felt that it was rash to make a methodology change when things may go back to normal in the future. Another respondent requested that the Commission consults on this topic again once an alternative methodology has been proposed.

Our position

We will explore more future proof methodologies for the ongoing measurement of gambling participation and prevalence which are able to withstand the threats that are posed to more traditional methodologies.

It is very important to maintain the robustness and accuracy of the research. Therefore, in commissioning a new survey, we will aim to use a random probability sampling approach if possible.

In commissioning a new survey, we will be open to both fully online and ‘push to web’ approaches (supplemented by offline data collection for periodic benchmarks) and mixed methodology approaches which will maximise sample representativeness.

We will continue to build our knowledge of other national surveys that have switched methodology to build on best practice.

We acknowledge the suggestion that our research programme should utilise a wider range of research methodologies. Whilst we do not think that all of these are suitable to produce official statistics about gambling participation and prevalence as per this consultation, subject to budget we continue to be open to exploring new and innovative research methodologies in our wider programme of research. We also welcome the use of new and varied approaches by partners conducting research in support of the National Strategy for Reducing Gambling Harm and will continue to ensure that learnings from the wider research space are integrated into our evidence base.

Previous section
Proposal 4: Frequency and turnaround time Next section
Proposal 6: Survey pilot
Is this page useful?
Back to top