Report
Gambling participation and the prevalence of problem gambling survey: Experimental statistics stage
Gambling Commission report produced by NatCen on the experimental statistics stage of the gambling participation and the prevalence of problem gambling survey.
Contents
- Executive summary
- Introduction
- Methodology and response
- Testing an alternative approach to the selection of participants within households
- Introduction
- Completion rates in responding households
- Adherence to participant-selection instructions
- Duplicate cases
- Household clustering
- Prevalence of gambling behaviours
- Conclusion
- Measuring gambling-related harms
- Testing different approaches to asking questions about gambling participation
Additional considerations for the postal questionnaire
You can view tables referenced in this section by downloading the file Tables A1 to A48 - Gambling Survey - Experimental statistics stage (XLSX)
Completion of postal questionnaires relies on correct navigation by participants. Whilst routing instructions are provided, there is no constraint on which questions are completed nor on the answers that can be given and hence data is subject to errors when routing instructions are not followed. These type of errors are not possible in the online questionnaire, where the routing is handled by the questionnaire program.
The three sets of gambling participation questions took up the same amount of space (nine pages) in the postal questionnaire. However, the layout, routing instructions and order of follow-up questions differed, and this could have impacted on how participants answered the questions.
For the long-list and hierarchical-list approaches participants were routed to later pages to record further detail about each activity recorded. To an extent, this relied on participants remembering which activities they initially recorded and going to the right page and activity to record the further detail. The chunked-list approach had less complex routing instructions with follow-up questions following directly after each activity type.
Adherence to routing instructions for Problem Gambling Severity Index (PGSI) items
Those who had gambled in the last 12 months were routed to the PGSI items. Those who had not gambled in the last 12 months were routed past the PGSI items.
Looking at those who had gambled in the last 12 months who were routed to but did not answer the first PGSI item, non-response was 37 percent for the long-list, 31 percent for the hierarchical-list and 21 percent for the chunked-list approach. The same pattern and similar proportions were seen for the final PGSI item where non-response was 38 percent, 34 percent and 24 percent respectively. All approaches had higher levels of item non-response to the PGSI items than was seen in the pilot and in step 1.
The highest level of non-response for the long-list approach is likely because the routing instructions were more complex compared with the chunked-list approach, leading to more participants missing the questions (Figure 11: Postal questionnaire PGSI non-response amongst those who had gambled in the last 12 months, by questions approach as follows).
Figure 11: Postal questionnaire Problem Gambling Severity Index (PGSI) non-response amongst those who had gambled in the last 12 months, by questions approach
| Routing error at PGSI1: Bet more than could really afford to lose | Gambling participation questions approach: Long-list (percentage) |
Gambling participation questions approach: Hierarchical-list (percentage) |
Gambling participation questions approach: Chunked-list (percentage) |
|---|---|---|---|
| Proportion of those who had gambled in the last 12 months that did not answer the question | 37% | 31% | 21% |
| Bases (unweighted): those who had gambled in the last 12 months (number) | 264 | 234 | 227 |
| Routing error at PGSI9: Felt guilty about the way gamble or what happens when gamble | Gambling participation questions approach: Long-list (percentage) |
Gambling participation questions approach: Hierarchical-list (percentage) |
Gambling participation questions approach: Chunked-list (percentage) |
|---|---|---|---|
| Proportion of those who had gambled in the last 12 months that did not answer the question | 38% | 34% | 24% |
| Bases (unweighted): those who had gambled in the last 12 months (number) | 264 | 234 | 227 |
Figure 11 information
Note: The table shows the percentage of respondents who were eligible to answer the question but did not answer it across the three different versions of the questionnaire, so the responses shown will not add up to 100 percent.
Those who had not gambled in the last 12 months should not have answered the PGSI items. The proportion who nonetheless did so was highest in the chunked-list approach (44 percent) and lower in the hierarchical and long-list approaches (11 percent and 18 percent respectively). These differences were statistically significant. This lower ‘routing error’ rate on the long-list and hierarchical-list approaches suggests that the instructions were clearer and easier to navigate for those who had not gambled in the last 12 months ('Figure 13: Postal completions: proportion of those who had not gambled in the last year with PGSI routing errors, by questions approach' as follows).
Figure 12: Postal completions: proportion of those who had not gambled in the last year with Problem Gambling Severity Index (PGSI) routing errors, by questions approach
| Routing error at PGSI1: Bet more than could really afford to lose | Gambling participation questions approach: Long-list (percentage) |
Gambling participation questions approach: Hierarchical-list (percentage) |
Gambling participation questions approach: Chunked-list (percentage) |
|---|---|---|---|
| Proportion of those who had not gambled in the last 12 months that answered question | 18% | 11% | 44% |
| Bases (unweighted): Those who had not gambled in the last 12 months (number) | 199 | 220 | 176 |
| Routing error at PGSI9: Felt guilty about the way gamble or what happens when gamble | Gambling participation questions approach: Long-list (percentage) |
Gambling participation questions approach: Hierarchical-list (percentage) |
Gambling participation questions approach: Chunked-list (percentage) |
|---|---|---|---|
| Proportion of those who had not gambled in the last 12 months that answered question | 17% | 10% | 39% |
| Bases (unweighted): those who had gambled in the last 12 months (number) | 199 | 220 | 176 |
Figure 12 information
Note: The table shows the percentage of respondents who answered the question even though they had not gambled in the past 12 months across the three different versions of the questionnaire, so the responses shown will not add up to 100 percent.
Regression modelling was carried out to look at the effects of the demographic variables used in the weighting strategy on the first and last PGSI items, PGSI1 and PGSI9. The aim was to see the relative impact of the weighting variables compared with the impact of the question approach (that is whether the participant was asked the long-list, hierarchical-list or chunked-list set of gambling participation questions). The independent variables included were, from the weighting strategy: sex, age group, ethnicity, household income (grouped), tenure and highest education level and also, the question approach. The dependent variables in the analysis were PGSI1 and PGSI9.
When accounting for the possible effect of variables used in weighting, the proportion of non-response to PGSI1 was lower in the chunked-list approach than for the other two approaches but there was no difference between the approaches for PGSI9. Hence, question approach had a significant impact on response which confirms the initial analyses (Tables A.43: Multivariate analysis with PGSI1 and A.44: Multivariate analysis with PGSI9).
Collecting detail about gambling activities in the last four weeks
When participants reported that they had gambled (that is spent money) on a certain activity in the last four weeks they were instructed to answer a set of follow-up questions on that activity. For the long-list and hierarchical-list approaches postal participants were first asked what gambling activities they participated in and then routed to later pages to record further detail. This relied on participants remembering which activities they initially recorded and going to the right page and activity to record the further detail.
The chunked-list approach had less complex routing instructions with follow-up questions following directly after each activity type. However, for some activities, the routing instructions in the hierarchical-list better matched the category listed at the previous question. As an example, the lotteries follow-up questions in all approaches began with an instruction of 'If you bought lottery tickets in the last four weeks answer the following questions', an instruction that was likely clearer and easier to navigate for hierarchical-list participants as it matched the category previously listed (in the other two approaches ‘lotteries’ was separated into two categories so there was not a direct match).
There was a lower non-response to the first lotteries follow-up question amongst hierarchical-list approach participants (21 percent compared with 30 percent in the long-list approach and 32 percent in the chunked-list approach). A lower proportion of participants who had not bought a lottery ticket in the last four weeks also mistakenly answered these questions (14 percent). None of these differences reached statistical significance (Table A.45 Routing errors to first lottery follow-up question, postal completions).
Missing responses
The long-list and hierarchical-list approaches asked participants to code 'yes' or 'no' for each listed activity. The chunked-list approach had multi code 'tick all that apply' answer options (with 'None of these' if they had not participated in any gambling activity during the recording period).
There was little difference between the long-list and hierarchical-list approaches in terms of non-response (between two percent and six percent). For the chunked-list, non-response was higher with between 11 percent and 27 percent of participants not selecting any of the activities nor the 'None of these' answer option.
This suggests that the yes or no grids used on the long and hierarchical-lists worked better than the multi-coded question in the chunked-list approach. It should also be borne in mind that switching between time periods in the chunked-list approach may have confused participants and confounded the rate of non-response (Table A.46 Missing responses to gambling participation questions, postal completions).
Previous sectionPotential primacy effect with the long-list approach Next section
Other considerations
Last updated: 22 August 2023
Show updates to this content
Formatting changes