Report
Gambling participation and the prevalence of problem gambling survey: Experimental statistics stage
Gambling Commission report produced by NatCen on the experimental statistics stage of the gambling participation and the prevalence of problem gambling survey.
Contents
- Executive summary
- Introduction
- Methodology and response
- Testing an alternative approach to the selection of participants within households
- - Introduction
- - Completion rates in responding households
- - Adherence to participant-selection instructions
- - Duplicate cases
- - Household clustering
- - Prevalence of gambling behaviours
- - Conclusion
- Measuring gambling-related harms
- Testing different approaches to asking questions about gambling participation
Introduction
One of the original aims of the survey was to review and refresh the gambling participation question commonly used on surveys, such as the Health Survey for England (HSE) 2018: Survey documentation (opens in new tab). The intention was to better reflect the current diversity of gambling products and to facilitate analysis of problem gambling prevalence at a product level.
One of the proposals that was consulted on by the Gambling Commission at the beginning of this project was to review the way respondents are asked about their participation in different gambling activities within the gambling participation and prevalence research consultation. The pilot stage of the project then began with gathering ideas from stakeholders about how best to update and refresh the list of activities asked about, and conducting cognitive testing to explore the ways in which people understood the descriptions of gambling activities used in survey questions, identifying any misunderstandings, ambiguities and missing activities.
Step 2 of the experimental statistics stage then aimed to assess the impact of asking questions about gambling participation in different ways on survey estimates of gambling participation rates.
Three different ways for asking questions about gambling activities participated in over the last 12 months and the last four weeks were assessed:
- a long-list approach
- a hierarchical-list approach
- a chunked-list approach.
The two time frames were asked about in order to capture recent activity (such as in the last four weeks) and also, less frequent gambling activity (such as over the last 12 months). The 12 months’ time frame was also used to correctly route into the Problem Gambling Severity Index (PGSI) items.
Survey participants were randomly assigned to one approach. For all three approaches, participants were asked whether they had spent money on any of the listed activities in the past 12 months. If they said yes, they were then asked the same question (and presented with the same answer codes) but about the last four weeks.
The wording of two questions was the same for all three approaches, but the answer options differed as follows.
Long-list approach
For the long-list approach, participants were presented with a redefined version of the list of activities used on the HSE, which distinguished between taking part in activities online and in-person. A total of 20 answer options, including ‘other’, were presented and participants were instructed to select 'yes' or 'no' for each activity.
Hierarchical-list approach
For the hierarchical-list approach, participants were presented with a list of 10 high-level activities; more granular-level activities then sat within the overarching high-level categories. As with the long-list approach, participants were instructed to select 'yes' or 'no' for each activity.
Chunked-list approach
For the chunked-list approach, the long-list was combined into five different categories or 'chunked' lists in which participants were asked about each group sequentially. Participants were asked to select all that applied (or ‘none of these’ if none of the five categories applied).
For each approach, those who had participated in a particular gambling activity in the last four weeks were asked a series of follow-up questions. Chunked-list participants were asked a shorter list of similar in nature activities at a time and if they had not spent money on the listed activities, they were routed to the next category of activities. For the long-list and hierarchical-list approaches, additional follow-up questions were asked including those to distinguish between online and offline participation and to capture frequency of play and spend. The questionnaire specifications are provided in Appendix D - Step 2 web questionnaire (PDF) and Appendix E - Step 2 paper questionnaires (PDF).
For step 2, 14,982 addresses were issued with the aim of achieving 4,000 productive individual questionnaires. The sampled addresses were randomly allocated to one of three equal sized groups.
Random allocation was used to eliminate the potential for systematic self-selection bias. Minimising differences between participants in each group that could be responsible for any differences seen in estimates between the three question approaches ensures that any observed differences are due to the way the question was asked17. In total, 4,994 addresses were issued to each of the survey approaches with the aim of achieving 1,333 productive individual questionnaires from each.
A total of 3,492 individual questionnaires were completed:
- 1,230 in the long-list approach group (35 percent of completions)
- 1,184 in the hierarchical-list approach group (34 percent of completions)
- 1,078 in the chunked-list approach group (31 percent of completions).
References
17 Experiments for Evaluating Survey Questions, Jon Krosnick (2011). In Question Evaluation Methods: Contributing to the Science of Data Quality. Hoboken, New Jersey; John Willey & Sons; pp. 215-238, Jennifer Madans, Kristen Miller, Aaron Maitland, Gordon Willis [Eds].
Comparison of gambling participation across the three approaches
Last updated: 18 April 2023
Show updates to this content
No changes to show.