Report

Gambling participation and the prevalence of problem gambling survey: Experimental statistics stage

Gambling Commission report produced by NatCen on the experimental statistics stage of the gambling participation and the prevalence of problem gambling survey.

---

Published
19 April 2023

Last updated
22 August 2023 - Changes

Document actions
Print or save

Contents

• Gambling participation and the prevalence of problem gambling survey: Experimental statistics stage

---

• Executive summary
• - Experimentation work
• - Response
• - Assessment of experiment results
• - Recommendations
• Introduction
• - Background
• - Aims and overview
• Methodology and response
• - Survey design
• - Questionnaire content and design
• - Response to the survey
• Testing an alternative approach to the selection of participants within households
• - Introduction
• - Completion rates in responding households
• - Adherence to participant-selection instructions
• - Duplicate cases
• - Household clustering
• - Prevalence of gambling behaviours
• - Conclusion
• Measuring gambling-related harms
• - Introduction
• - Testing different ways of asking about gambling harms
• - Results
• - Conclusion
• Testing different approaches to asking questions about gambling participation
• - Introduction
• - Comparison of gambling participation across the three approaches
• - Potential primacy effect with the long-list approach
• - Additional considerations for the postal questionnaire
• - Other considerations
• - Conclusion

Potential primacy effect with the long-list approach

You can view tables referenced in this section by downloading the file Tables A1 to A48 - Gambling Survey - Experimental statistics stage (XLSX)

Questions with a long-list of answer options are known to be at greater risk of a type of systematic error called primacy bias when presented visually¹⁸. Response options visually presented at the start of a long-list are more likely to be selected than those further down the list. One method of combatting this is to break up longer lists into chunks, hence the inclusion of this approach in the experiment. It might be expected that if a primacy effect is evident in the long-list approach, gambling activities further down the list would be selected by a smaller proportion of participants compared with the shorter chunked-list approach.

Another method to combat the risk of primacy bias is to require participants to answer yes or no to each item presented in the list; this was used in the long-list approach. Randomising the list would have been another method but whilst this would have been possible in the online questionnaire, it could not be done in the postal version.

The analysis compared rates of gambling participation for the long-list and chunked-list approaches. The hierarchical-list approach was not included due to the different way activities were presented to participants.

Rates of gambling participation for the two approaches, the long-list and chunked-list, were similar across most gambling activities. The greatest differences was seen for activities nearer the top of the list: National Lottery scratchcards (24 percent for the long-list and 18 percent for the chunked-list, a statistically significant difference), tickets for other charity lotteries (23 percent and 18 percent respectively, not statistically significant), and National Lottery online instant win games (10 percent and five percent respectively a statistically significant difference).

Activities in the latter half of the participation list differed by up to three percentage points across the two approaches. This as well as the higher endorsement of activities overall suggests that the long-list was not subject to primacy bias.

A similar (but again, not statistically significant) pattern and absence of evidence of primary bias was found for gambling activities in the last four weeks. This may be because the use of yes or no response options in the long-list was more effective at encouraging participants to engage with each item on the list, than chunking the list (Table A.41: Gambling activities spent money on in the last 12 months, by questions approach; Table A.42: Gambling activities spent money on in the last four weeks, by questions approach).

References

¹⁸ See for example: Retrospective reports: The impact of response alternatives, Norbert Schwarz, Hans-J. Hippler, Elisabeth Noelle-Neumann (1994). In Autobiographical memory and the validity of retrospective reports, New York,: Springer-Velag, pp187-202, Norbert Schwarz and Seymour Sudman [Eds].

Previous section
Comparison of gambling participation across the three approaches Next section
Additional considerations for the postal questionnaire

---

Last updated: 22 August 2023

Show updates to this content

Formatting changes