Statistics and research release
Developing survey questions to collect better data on gambling-related harms
Further work has been conducted to develop and test a series of survey questions aimed at collecting better data on the experience of gambling-related harms.
Additional data sets in this series
A summary of this release is as follows:
- the Gambling Commission has been piloting new survey questions on gambling-related harms
- the questions have been developed iteratively and have been incorporated into the wider pilot of a new approach to collecting participation and prevalence data
- this release provides a further update, analysis of the latest pilot data, recommendations from external experts, and next steps.
The Gambling Commission’s statutory role is to ensure licence holders are acting to protect children and other vulnerable people from being harmed or exploited by gambling. To do this, we need to understand the ways and extent that gambling consumers and others experience gambling harms.
The Commission has been piloting a new set of survey questions designed to understand the incidence, nature and severity of harm experienced as a result of someone’s own gambling (‘harms to self’), or someone else’s gambling (‘harms from others’). Initially, the questions were iteratively tested on our online tracker survey on a quarterly basis, and we published an update on the pilot of survey questions to understand gambling-related harm from this stage at the end of 2021. The questions were then refined and tested again as part of the pilot of our new approach to collecting participation and prevalence data. The process was also reviewed by questionnaire design experts.
Our aim is for the harms questions to be asked alongside core questions on participation and problem gambling in 2023, and become part of our suite of official statistics.
We are not aiming to develop a headline score or scale of gambling-related harms, or measure the cost of gambling harms to society. The Commission has a role to play in utilising its existing surveys to add to the wider evidence base. However, there is still a need for a range of partners to be involved in the funding and delivery of research to measure and understand gambling harms and the impact that they have on individuals, families, communities, and society.
Definition of gambling-related harms
Gambling-related harms are the adverse impacts from gambling on the health and wellbeing of individuals, families, communities, and society.
These harms are diverse, affecting resources, relationships and health, and may reflect an interplay between individual, family and community processes. The harmful effects from gambling may be short-lived but can persist, having longer-term and enduring consequences that can exacerbate existing inequalities.
Measuring gambling-related harms - a framework for action (PDF) (opens in a new tab) provides more information.
Following the previous update, the initial question set has been refined and tested further as part of the pilot of our new approach to collecting participation and prevalence data. The questions were reviewed by questionnaire development experts at NatCen’s Questionnaire Development and Testing Hub before the pilot commenced in January 2022.
The pilot data were then analysed by academics at the University of Glasgow to assess their performance and make recommendations for future development.
In addition, two external experts, Robert Williams and Rachel Volberg, were asked to review the process undertaken to date and provide independent, expert advice on how the questions could be modified or the approach strengthened.
This release contains a summary of the findings and recommendations from these two pieces of work, along with a further update on how the pilot has progressed since then and next steps for this work.
Overall, the external review and analysis of the pilot data suggested that the question development process was robust, and the questions were clear and unambiguous. The following improvements were suggested:
- Include a broader range of harm issues in questions
The range of harms included in the question set were heavily weighted towards financial harms. The Gambling Commission should review their prior pilot data to establish a more balanced set of questions, covering a broader range of issues.
- Change scaled response options to have more equally spaced responses
The pilot data showed that questions which used scaled answer options were not performing in the way expected, with fewer people reporting experiencing each harm ‘a little’ than ‘a lot’. The scaling between answer options was also uneven. The Commission should conduct experimental work comparing results between harms questions using binary 'yes' and 'no' answer options and (revised) scaled answer options.
- Consider how scaled answer options are reported
Relatedly, if scaled answer options are retained, the Commission will need to consider how it reports summary statistics from these questions and how these are defined.
- Revise the filter question into the ‘harms from others’ section
The filter question into the ‘harms from others’ questions should be revised. The current question appears to under-report the number of people who know people close to them who gamble. This will then lead to underestimation of harms from others.
- Change ordering of the harms questions
The ordering of the harms questions should be changed so that items relating to less severe harms are presented first.
- Review wording of all harms questions
The question wording of all harms questions should be further reviewed by questionnaire development experts.
- Continue collecting data on suicide ideation and attempted suicide
Methodologically, it is encouraging that there was some endorsement of the gambling-related suicide question within the pilot. This should be included in the next phase of development work to assess further.
Further progress and next steps
Since this review we have taken the following action:
- worked with NatCen to broaden the range of harms covered in the questions
- developed an experiment to determine whether a binary response or a scaled option (with revised scales) works best when asking about experiences of harm
- revised the filter question into the ‘harms from others’ section as part of this experiment and will assess its effectiveness
- the ordering and wording of the harms questions have been reviewed by experts at NatCen
- the questions on suicide ideation and attempted suicide have been retained for the experimental phase, providing additional data to allow us to assess these questions further.
Our next steps
Our next steps will be to:
- review the results of the experimental phase and robustly evaluate which approach works best
- develop our approach to analysing the data and engaging with stakeholders on how we communicate the headline statistics and findings.
Full findings and recommendations
View the full note of findings and recommendations for developing survey questions capturing gambling-related harms.
Data and downloads
There are no files for this release.
We are always keen to hear how these statistics are used and would welcome your views on this publication.