Cookies on the Gambling Commission website

The Gambling Commission website uses cookies to make the site work better for you. Some of these cookies are essential to how the site functions and others are optional. Optional cookies help us remember your settings, measure your use of the site and personalise how we communicate with you. Any data collected is anonymised and we do not set optional cookies unless you consent.

Set cookie preferences

You've accepted all cookies. You can change your cookie settings at any time.

Skip to main content
  1. Home
  2. About us
  3. Gambling participation and the prevalence of problem gambling survey: Experimental statistics stage

Report

Gambling participation and the prevalence of problem gambling survey: Experimental statistics stage

Gambling Commission report produced by NatCen on the experimental statistics stage of the gambling participation and the prevalence of problem gambling survey.

Published
19 April 2023

Last updated
18 April 2023 - Changes

Document actions
Print or save

Contents

Recommendations

Household selection

There was no discernible experimental condition effect on household response rates, duplications nor gambling participation rates. There was evidence of significant clustering of gambling behaviours among households with three or four participants. As this can impact on the accuracy of the gambling participation data our recommendation is that inviting up to two adults per household to take part, is the preferred option going forward.

Within-household selection bias is an issue across push-to-web survey methodologies (for example, see Community Life Survey: Investigating the feasibility of sampling all adults in the household (opens in new tab)). However, the current gambling survey weighting strategy addresses this by modelling the number of responses per household (for households with more than one eligible adult). This step in the weighting strategy tests a range of variables for association with a number of key survey responses, including the number of eligible adults in the household, to control for bias arising from within-household non-response. In the next phase, we recommend that analysis is carried out to identify if patterns in gambling behaviour are different between those households where no household selection took place and those where there was a within-household selection.

It is important to note that weights do not overcome the bias towards those who gamble identified in the pilot. To do this, one of two sources of information would be needed, either the true number of gamblers in a household or an external source for the number of gamblers which could be calibrated to. As neither of these are available, the focus for reducing the bias is on making the invite and reminder letters more appealing to those who do not gamble.

Those responding 'yes' in condition B (binary answer option) produced similar rates of participants classified as experiencing harms to those responding 'fairly often' or 'very often' in condition A. Those answering 'yes' on the harms from others questions displayed less consistent patterns of association with personal wellbeing, contrary to expectations. For these reasons combined, we recommend retaining the refined four-point answer scale for the next step of the survey.

When it comes to analysing the data from the scaled answer options, the Gambling Commission will need to carefully assess the extent to which endorsement represents the experience of harm. We concur with Williams and Volberg2 that those answering 'occasionally' are most likely representing the potential for harm rather than experience of it. If the Commission wanted to explore this further, qualitative follow-up interviews could be undertaken with participants to this survey to explore their experiences.

Further recommendations related to the analysis of gambling harms are:

  • the changes to the screening questions to route people into the harms from others questions have made improvements and should be retained
  • in the harms from others questions, the answer options for questions asking about harms to health, borrowing money or feeling guilty and/or embarrassed or ashamed should match the answer options for the other questions
  • the Commission should consider how it wants to analyse and present the gambling suicidality questions and think about steps to reduce analytical ambiguity
  • if the harms to self questions are ever included in a survey which is not also using the Problem Gambling Severity Index (PGSI) instrument, answer options to these three items should also be changed for consistency.

Gambling participation

No clear picture emerged as to which of the three approaches (long-list, chunked-list and hierarchical-list) performed best in capturing information about gambling participation. Whilst all three approaches had limitations, the limitations of the hierarchical-list and chunked-list approaches were considered more problematic than those of the long-list approach.

On balance, the recommendation was that the long-list approach should be used going forward but that the structure of the postal version of the questionnaire and the routing instructions within it should be reviewed and where possible improved.

The impact of changes to the postal questionnaire should be assessed by comparing results of the next phase of data collection with the data from this experiment. Specifically, the goal would be to see a reduction in the levels of item non-response to follow-up questions in the postal questionnaire.

Use of QR codes to access the online survey

In step 2, two Quick Response (QR) codes were included in the invitation letter and reminders. The QR codes, when scanned, took participants straight into the online questionnaire, bypassing the need to manually enter any access information.

Uptake of the QR codes was high: 52 percent of those completing the survey online did so via the QR code though usage decreased with increasing age.

It is recommended that QR codes are retained as an alternative way for participants to access the survey in future phases.

References

2 Developing survey questions capturing gambling-related harms - Part A - Expert review of question development by Professor Robert Williams and Dr Rachel Volberg, Heather Wardle, Viktorija Kesaite, Robert Williams, Rachel Volberg (2022).

Previous section
Assessment of experiment results
Is this page useful?
Back to top