Report

Gambling participation and the prevalence of problem gambling survey: Experimental statistics stage

Gambling Commission report produced by NatCen on the experimental statistics stage of the gambling participation and the prevalence of problem gambling survey.

---

Published
19 April 2023

Last updated
18 April 2023 - Changes

Document actions
Print or save

Contents

• Gambling participation and the prevalence of problem gambling survey: Experimental statistics stage

---

• Executive summary
• - Experimentation work
• - Response
• - Assessment of experiment results
• - Recommendations
• Introduction
• - Background
• - Aims and overview
• Methodology and response
• - Survey design
• - Questionnaire content and design
• - Response to the survey
• Testing an alternative approach to the selection of participants within households
• - Introduction
• - Completion rates in responding households
• - Adherence to participant-selection instructions
• - Duplicate cases
• - Household clustering
• - Prevalence of gambling behaviours
• - Conclusion
• Measuring gambling-related harms
• - Introduction
• - Testing different ways of asking about gambling harms
• - Results
• - Conclusion
• Testing different approaches to asking questions about gambling participation
• - Introduction
• - Comparison of gambling participation across the three approaches
• - Potential primacy effect with the long-list approach
• - Additional considerations for the postal questionnaire
• - Other considerations
• - Conclusion

Assessment of experiment results

In step 1, the analysis looked to establish whether there were significant differences between two experimental conditions – where a maximum of two or a maximum of four adults invited to take part. There was no discernible experimental condition effect on household response rates, duplications nor gambling participation rates. There was evidence of significant clustering of gambling behaviours among households with three or four participants. As this can impact on the accuracy of the gambling participation data the recommendation was that going forward, up two adults per household should be invited to take part in the survey.

A further experiment in step 1 looked at gambling-related harms questions. Those responding 'yes' in condition B (binary answer option) produced similar rates of participants classified as experiencing harms to those responding 'fairly often' or 'very often' in condition A (scaled answer options). Those answering 'yes' on the harms from others questions displayed less consistent patterns of association with personal wellbeing, contrary to expectations. For these reasons combined, we recommend retaining the refined four-point answer scale for the next step of the survey.

When it comes to analysing the data from the scaled answer options, the Gambling Commission will need to carefully assess the extent to which endorsement represents the experience of harm. We concur with Williams and Volberg¹ that those answering 'occasionally' are most likely representing the potential for harm rather than experience of it. If the Commission wanted to explore this further, qualitative follow-up interviews could be undertaken with participants to this survey to explore their experiences.

Step 2 aimed to assess the impact of asking questions about gambling participation in different ways on survey estimates of gambling participation rates. The following three different ways of asking questions about gambling activities participated in over the last 12 months and the last four weeks were assessed:

• long-list approach
• chunked-list approach
• hierarchical-list approach.

No clear picture emerged as to which of the three approaches (long-list, chunked-list and hierarchical-list) performed best in capturing information about gambling participation. Whilst all three approaches had limitations, those of the hierarchical-list and chunked-list approaches were considered more problematic than those of the long-list approach.

On balance, the recommendation was that the long-list approach should be used going forward but that the routing instructions and visual layout of the postal version should be improved. The impact of changes to the postal questionnaire should then be assessed by comparing results of the next phase of data collection with the data from this experiment. Specifically, the goal would be to see a reduction in the levels of item non-response to follow-up questions in the postal questionnaire.

In step 2, two Quick Response (QR) codes were included in the invitation letter and reminders. The QR codes, when scanned, took participants straight into the online questionnaire, bypassing the need to manually enter any access information. Uptake of the QR codes was high: 52 percent of those completing the survey online did so via the QR code though usage decreased with increasing age.

References

¹ Developing survey questions capturing gambling-related harms - Part A - Expert review of question development by Professor Robert Williams and Dr Rachel Volberg, Heather Wardle, Viktorija Kesaite, Robert Williams, Rachel Volberg (2022).

Previous section
Response Next section
Recommendations

---

Last updated: 18 April 2023

Show updates to this content

No changes to show.