Gambling participation and the prevalence of problem gambling survey: Experimental statistics stage
Gambling Commission report produced by NatCen on the experimental statistics stage of the gambling participation and the prevalence of problem gambling survey.
- Executive summary
- Methodology and response
- Testing an alternative approach to the selection of participants within households
- Measuring gambling-related harms
- Testing different approaches to asking questions about gambling participation
Potential primacy effect with the long-list approach
You can view tables referenced in this section by downloading the file Tables A1 to A48 - Gambling Survey - Experimental statistics stage (XLSX)
Questions with a long-list of answer options are known to be at greater risk of a type of systematic error called primacy bias when presented visually18. Response options visually presented at the start of a long-list are more likely to be selected than those further down the list. One method of combatting this is to break up longer lists into chunks, hence the inclusion of this approach in the experiment. It might be expected that if a primacy effect is evident in the long-list approach, gambling activities further down the list would be selected by a smaller proportion of participants compared with the shorter chunked-list approach.
Another method to combat the risk of primacy bias is to require participants to answer yes or no to each item presented in the list; this was used in the long-list approach. Randomising the list would have been another method but whilst this would have been possible in the online questionnaire, it could not be done in the postal version.
The analysis compared rates of gambling participation for the long-list and chunked-list approaches. The hierarchical-list approach was not included due to the different way activities were presented to participants.
Rates of gambling participation for the two approaches, the long-list and chunked-list, were similar across most gambling activities. The greatest differences was seen for activities nearer the top of the list: National Lottery scratchcards (24 percent for the long-list and 18 percent for the chunked-list, a statistically significant difference), tickets for other charity lotteries (23 percent and 18 percent respectively, not statistically significant), and National Lottery online instant win games (10 percent and five percent respectively a statistically significant difference).
Activities in the latter half of the participation list differed by up to three percentage points across the two approaches. This as well as the higher endorsement of activities overall suggests that the long-list was not subject to primacy bias.
A similar (but again, not statistically significant) pattern and absence of evidence of primary bias was found for gambling activities in the last four weeks. This may be because the use of yes or no response options in the long-list was more effective at encouraging participants to engage with each item on the list, than chunking the list (Table A.40: Gambling activities spent money on in the last 12 months, by questions approach; Table A.41: Gambling activities spent money on in the last four weeks, by questions approach).
18 See for example: Retrospective reports: The impact of response alternatives, Norbert Schwarz, Hans-J. Hippler, Elisabeth Noelle-Neumann (1994). In Autobiographical memory and the validity of retrospective reports, New York,: Springer-Velag, pp187-202, Norbert Schwarz and Seymour Sudman [Eds].
Comparison of gambling participation across the three approaches Next section
Additional considerations for the postal questionnaire
Last updated: 18 April 2023
Show updates to this content
No changes to show.