Experiment 1 – Household selection and presentation of harms statements
NatCen explored the best ways to encourage both gamblers and non-gamblers to complete the survey to ensure a representative spread of respondents.
They changed the wording on the invitation letters to make it clearer that we are interested in the responses from both gamblers and non-gamblers. NatCen also conducted a split sample experiment to find the optimum approach for the number of adults per household that we invite to take part in the survey, the experiment compared results when inviting 2 adults versus up to 4 adults per household.
Within this first experimental phase, NatCen also continued to build on the work that the Gambling Commission have been doing to develop a way of understanding the incidence and nature of harms associated with gambling. As part of this work, they ran an experiment to determine whether a binary response (yes or no) or a scaled option (four-point answer scale) works best when asking about experiences of harm.
NatCen’s recommendations from Experiment 1 were as follows:
- the preferred option is to invite up two adults per household to take part
- the refined four-point answer scale for the harms questions should be retained ('very often', 'fairly often', 'occasionally' and 'never')
- changes to the screening questions to route people into the harms from others questions have made improvements and should be retained
- in the harms from others questions, the answer options for questions asking about harms to health, borrowing money or feeling guilty and/or embarrassed or ashamed should match the answer options for the other harms questions.
Experiment 2 – Construction and presentation of the gambling activity list
In the second experiment, NatCen explored the construction and presentation of the gambling activity list by testing an updated list of gambling activities as well as testing different ways of presenting the list of gambling activities to respondents. The following three ways of presenting the gambling participation list were tested:
- long-list approach
- chunked-list approach
- hierarchical-list approach.
Finally, in the second experiment NatCen also tested the inclusion of a Quick Response (QR) code to make it easier for those completing the survey online. Two QR codes were included in the invitation letter and reminders, which, when scanned, took respondents straight into the online questionnaire, bypassing the need to manually enter any access information.
NatCen’s recommendations from Experiment 2 were as follows:
- the long-list approach for asking about gambling participation should be used going forward
- routing instructions for the postal version of the questionnaire should be reviewed and where possible, improved. Particular attention should be given to the routing instructions into the Problem Gambling Severity Index (PGSI) screen, to reduce non-response
- QR codes should be retained as an alternative way for participants to access the survey in future phases.
The next stage of the project is Step 3 of the experimental stage. Step 3 is a final test of the agreed approach. This step involves taking the learnings from Experiments 1 and 2, applying them and ensuring that the survey design and questionnaire content is robust and fit for official statistics continuous data collection. The Step 3 experiment will take place between April and July 2023.
The Gambling Commission have also commissioned some additional qualitative follow up interviews, to explore the experiences of those participants who said they had occasionally experienced gambling related harms. This work was a recommendation from Experiment 1 and will help us to understand responses in more detail, including whether those saying they have experienced harm occasionally are representing the potential for harm rather than experience of it. It will also explore the connections between different harms amongst participants who had experienced multiple harms.
Full publication and key information
View the Gambling participation and the prevalence of problem gambling survey: Experimental statistics stage report.
Publication produced by: NatCen.
Publication authors: Robert Ashford, Beverley Bates, Charlotte Bergli, Sierra Mesplie-Cowan (NatCen Social Research), Viktorija Kesaite (University of Glasgow) and Heather Wardle (University of Glasgow).
Responsible Statistician: Helen Bryce (Head of Statistics).