This box is not visible in the printed version.
Further work has been conducted to develop and test a series of survey questions aimed at collecting data on the experience of gambling-related harms.
Published: 19 October 2022
Last updated: 19 October 2022
This version was printed or saved on: 27 April 2024
Online version: https://www.gamblingcommission.gov.uk/about-us/guide/developing-survey-questions-capturing-gambling-related-harms
Overview: Heather Wardle1, Viktorija Kesaite1, Robert Williams2, Rachel Volberg3
Since 2020, the Gambling Commission has conducted work to develop and test a series of survey questions aimed at collecting data on the experience of gambling harms. This focused on capturing data on harms from your own gambling ('harms to self') and harms generated due to the gambling of others ('harms from others').
The principles guiding the Commission’s development process were that:
To do this, the Commission reviewed a range of different frameworks for gambling harms (which broadly map similar criteria though summarise them in different ways). They then created a suite of questions devised from the 72 harms statements developed by Browne et al. They chose 27 statements to pilot and test in three waves of their online omnibus survey and analysed the resulting data. This included looking at internal consistency of the data against other variables such as problem gambling score, gambling frequency, number of activities participated in etc.
Questions were repeated across waves to look at the stability of responses in repeated cross-sectional surveys. To reduce the number of items (because of respondent burden and limitations of questionnaire space), the Commission used factor analysis to reduce the number of statements to a smaller sub-set. This produced two questionnaire sub-sets, one measuring ‘harms to self’ and one measuring ‘harms from others’. They both consisted of 14 questions.
The next stage of development was to include these questions within the programme of work being undertaken by NatCen Social Research and the University of Glasgow to devise a new random probability survey capturing gambling prevalence across Great Britain (gambling prevalence survey). As part of the development work for this study, the harms questions were reviewed by questionnaire development experts at NatCen’s Questionnaire Development and Testing Hub, with suggested amendments made prior to their inclusion in the first pilot of the gambling prevalence survey.
To aid this process, two external experts, Robert Williams and Rachel Volberg were asked to review the Commission’s procedures undertaken to date and provide further expert advice on how the questions could be modified.
This note summarises the advice received from Robert Williams and Rachel Volberg (Part A), presents analysis of the harms questions from the pilot survey (Part B) and summarises the recommendations made to further test and improve the questions (Part C).
1 Gambling Research Glasgow, University of Glasgow
2 University of Lethbridge, Canada
3 Gemini Research, Mass, United States of America (USA)
Overall, Professor Robert Williams and Dr Rachel Volberg found the intent behind the question development to be excellent. They noted the importance of capturing fuller negative impacts of gambling compared with traditional problem gambling approaches4.
They found the process undertaken by the Gambling Commission to be thoughtful and logical. Furthermore, they indicated that draft instruments reviewed were based on a sound definition of gambling-related harm that is consistent with other definitions in the literature, for instance Browne et al. (2018)5, and that the instruments accurately capture the main harms experienced by most people being heavily weighted toward financial, relationship, and mental health harms (Browne et al., 20185; Browne et al, 20216; Shannon et al., 20177). They were also supportive of capturing gambling harms from someone else’s gambling.
They made the following suggestions for improvements.
Although the original list of harms was drawn from Browne et al’s original work, the organising framework to group harms together is that of Wardle et al. They recommended that when the final instrument is published the implications of this should be discussed.
They highlighted that a number of initiatives to measure harms are underway, with the Short Form Gambling Harm Scale (Browne et al., 2018)5 being the most dominant to date. They recommend that the Commission should be clear about how these instruments build on the prior work of Browne et al and be explicit about what the instruments potentially add (greater insights into the severity of harms experienced; measurement of harms to others). They also recommended comparing the correlation of the two instruments.
The instrument does not aspire to be psychometric scale. However, psychometric techniques have been used in development and especially in item reduction. This has produced an instrument that is heavily weighted towards the measurement of financial and/or resource harms. This, while in line with current literature (Langham et al, 2015)8, may also be an artefact of the analytical technique, as factor analysis favours items that are most often endorsed. They recommended reviewing the included items to ensure an equal balance across a range of theoretical domains of harms.
Positive endorsement of some items might not indicate the experience of harms but rather potential experience of harms. This is particularly pertinent to questions which have 'a little' as an answer option – it is unclear whether experiencing each harm a little represents a true negative consequence or not. The recommendation was made to remove this answer option or if it is to be retained, to be clear that this represents the 'potential' for harm, rather than experience of it.
Potentially the ‘harms to self’ questions could only be asked for those who gamble regularly, in order words in the last month, to reduce respondent burden and to minimise the likelihood of false positives.
4 See Appendix A for the questions reviewed
5 Validation of the Short Gambling Harm Screen (SGHS): A Tool for Assessment of Harms from Gambling (opens in a new tab)
6 The Evolution of Gambling-Related Harm Measurement: Lessons from the Last Decade (opens in a new tab)
7 Mapping the proportional distribution of gambling-related harms in a clinical and community sample (opens in a new tab)
8 Understanding gambling related harm: a proposed definition, conceptual framework, and taxonomy of harms (opens in a new tab)
In January 2022, NatCen Social Research and the University of Glasgow conducted a random-probability push-to-web pilot study to assess the impact of changing survey methodologies upon gambling participation and problem gambling prevalence rates.
Participants were first invited to complete the survey via an online survey, with postal questionnaires sent to those who did not complete online. In total, 1,078 participants aged 16 years and over, took part in the pilot study.
The gambling harms questions were included in this pilot9. Those who had gambled in the past 12 months were asked to report their experience of a range of individual harms (harms to self). This covered 14 individual harms, ranging from experience of violence and abuse to cutting back on spending on everyday items. These questions covered three broad domains:
Similar questions were asked of everyone who reported that they knew somebody close to them who had gambled. They were asked to think about the impact that somebody else’s gambling had upon them (harms from others). All participants were asked whether they had attempted suicide in the last 12 months and, if so, whether this was gambling-related.
For all harms questions, hidden skip codes were used in the online survey so that if someone attempted to move past the questions without completing, they were then presented with a 'prefer not to say' answer code. In the paper questionnaire, people could just leave questions blank.
These data were analysed to assess their performance and make recommendations for future development. Key results are summarised as follows.
Experience of severe harms defined as experience of violence and/or abuse; relationship breakdown; losing something of significant financial value; or gambling-related suicidality.
Response options for these questions were either yes or no.
Key results included:
Experience of other harms defined as ranging from cutting back on spending on everyday items to greater conflicts and/or arguments.
These questions were measured on a three-point scale: not at all; a little; a lot.
Key results included:
| Data definitions | A little (number of participants) |
A lot (number of participants) |
|---|---|---|
| Cutting back on spending on everyday items | 13 | 18 |
| Increased use of credit | 11 | 15 |
| Greater conflicts and/or arguments | 9 | 14 |
| Feeling isolated | 8 | 15 |
Correlation coefficients examined the relationship between experience of any financial harm; any health harm and any relationship harm with problem gambling status, finding a strong correlation (greater than 0.7) regardless of whether problematic gambling status was defined by the Problem Gambling Severity Index or the DSM-IV.
The following table entitled 'Whether anyone close to the participant gambling?' shows how many pilot participants reported that a family member, friend or someone else close to them gambled. Endorsement rates were highest for a family member (219 participants). Overall, 308 participants reported that someone close to them gambled and were thus eligible to complete the ‘harms from others’ gambling questions.
Overall, 65 percent of pilot participants reported gambling in the past year, with around 40 percent of participants living with someone else who gambled. However, when answering a specific question on this, only 308 (28.5 percent) participants reported that they knew someone close to them who had gambled. This suggests there is some under-reporting in this measure, which is likely related to how it is worded (asking about gambling with no clarification of the types of activities to include).
| Data definitions | Yes (percentage and number of participants) |
No (percentage and number of participants) |
|---|---|---|
| A family member | 20.33% (219) | 79.67% (858) |
| A friend | 8.73% (94) | 91.27% (983) |
| Someone else | 2.23% (24) | 97.77% (1053) |
Experience of severe harms defined as experience of violence and/or abuse; relationship breakdown; losing something of significant financial value; or gambling-related suicidality.
Response options for these questions were either yes or no.
Key results included:
Experience of other harms defined as ranging from cutting back on spending on everyday items to greater conflicts and/or arguments.
These questions were measured on a three-point scale: not at all; a little; a lot.
Key results included:
Overall, expert review and analysis of the harms questions within the pilot gambling prevalence survey suggested that question development process was robust and the questions presented to the survey participants were clear and unambiguous.
The resulting set of questions captured important domains of harms including financial, relationship, and health harms. It was particularly encouraging that similar numbers of men and women endorsed the harms questions.
As the development of these questions is an ongoing priority, the following improvements were suggested.
The range of harms included in the question set were heavily weighted towards financial harms.
The Commission should review their prior pilot data to establish a more balanced set of questions, covering a broader range of issues. For instance, questions related to productivity (for example, absenteeism or presenteeism at school and/or work), health (both physical and mental health), as well as other wellbeing measures could be added. This would give the resulting set of questions closer correspondence to the theoretical frameworks which underpin them.
The pilot data showed that questions which used scaled answer options were not performing in the way expected, with fewer people reporting experiencing each harm ‘a little’ than ‘a lot’.
The scaling between answer options is also uneven. Combined, this raises uncertainty about whether those experiencing something 'a little' are actually experiencing harms or rather experiencing the potential for harms. We recommend changing the scaled response options to have more equally spaced responses (if it is to be retained). We also recommend experimental work comparing results between harms questions using binary yes and no answer options and (revised) scaled answer options.
Currently, scaled answer options are used in the measurement of problem gambling which allows people to express endorsement without having to specify 'yes' in concrete terms. Retaining some flexibility in responses might be wise, given the stigma that surrounds gambling harms. However, this should be tested further.
Relatedly, if scaled answer options are retained, the Commission will need to consider how it reports summary statistics from these questions and how these are defined. For example, what level of endorsement most likely indicates that negative consequences are being incurred?
The filter question into the ‘harms from others’ questions should be revised. The current question appears to under-report the number of people who know people close to them who gamble. This will then lead to underestimation of harms from others.
The ordering of the harms questions should be changed so that items relating to less severe harms are presented first.
The question wording of all harms questions should be further reviewed by questionnaire development experts.
Methodologically, it is encouraging that there was some endorsement of the gambling-related suicide question within the pilot. This should be included in the next phase of development work to assess further.
Where harms were already included in either the DSM-IV (Diagnostic and Statistical Manual of Mental Disorders 4th Edition) instrument or the Problem Gambling Severity Index (PGSI), these questions were used and were not repeated.
Harms questions taken from the DSM-IV.
The following questions were provided:
The following options were provided:
The following questions were provided:
The following options were provided:
The following questions were provided:
The following options were provided:
The following questions were provided:
The following options were provided:
The following options were provided:
The following questions were provided:
The following options were provided:
The following questions were provided:
The following options were provided:
The following questions were provided:
The following options were provided:
The following questions were provided:
The following options were provided:
The following questions were provided:
The following options were provided:
The following questions were provided:
The following options were provided:
The following question was provided: To what extent, if at all, was this related to your or someone else’s gambling?
The following options were provided:
This section lists the body of evidence referred to in the development of survey questions capturing gambling-related harms to self and others. The links provided as follows and in the release are correct at the time of publication but will not be maintained over time.
Browne, M., Rawat, V., Tulloch, C., Murray-Boyle, C. and Rockloff, M., 2021. The evolution of gambling-related harm measurement: lessons from the last decade (opens in a new tab). International Journal of Environmental Research and Public Health, 18(9), p.4395.
Browne, M., Goodwin, B.C. and Rockloff, M.J., 2018. Validation of the Short Gambling Harm Screen (SGHS): A tool for assessment of harms from gambling (opens in a new tab). Journal of Gambling Studies, 34(2), pp.499-512.
Browne, M., Greer, N., Rawat, V. and Rockloff, M., 2017. A population-level metric for gambling-related harm (opens in a new tab). International Gambling Studies, 17(2), pp.163-175.
Langham, E., Thorne, H., Browne, M., Donaldson, P., Rose, J., and Rockloff, M., 2015. Understanding gambling related harm: a proposed definition, conceptual framework, and taxonomy of harms (opens in a new tab). BMC Public Health, 16(80).
Shannon, K., Anjoul, F. and Blaszczynski, A., 2017. Mapping the proportional distribution of gambling-related harms in a clinical and community sample (opens in a new tab). International Gambling Studies, 17(3), pp.366-385.
Wardle et al. (2018). Measuring gambling-related harms - A framework for action (opens in a new tab). Gambling Commission.