Report
Gambling participation and the prevalence of problem gambling survey: Experimental statistics stage
Gambling Commission report produced by NatCen on the experimental statistics stage of the gambling participation and the prevalence of problem gambling survey.
Contents
- Executive summary
- Introduction
- Methodology and response
- Testing an alternative approach to the selection of participants within households
- - Introduction
- - Completion rates in responding households
- - Adherence to participant-selection instructions
- - Duplicate cases
- - Household clustering
- - Prevalence of gambling behaviours
- - Conclusion
- Measuring gambling-related harms
- Testing different approaches to asking questions about gambling participation
Questionnaire content and design
The online mode was supplemented by a postal questionnaire follow up to enable less technologically literate people, those without internet access and those who preferred an alternative approach, to respond.
This step is essential for the new gambling survey as some gambling behaviours, notably the propensity to gamble online, is correlated to the probability to take part in an online survey and would therefore lead to biased results8. The pilot survey also demonstrated the importance of the postal option as 43 percent of completions were by this mode.
In addition to the questions being tested as part of the previously mentioned experiments, the questionnaires included content on:
- leisure activities, internet access and use
- online games and gambling adjacent activities (step 2 only)
- Problem Gambling Severity Index (PGSI)
- why people gamble and how it makes them feel (step 1 and online only for step 2)
- attitudes towards gambling (ATGS-8), gambling management tools and complaints (step 2 only)
- advertising and social media
- health and wellbeing, including general health, smoking and drinking status, impulsivity scale, Short Warwick-Edinburgh Mental Well-Being Scale (SWEMWBS) and suicidality questions
- demographic questions.
The online and postal questionnaires for step 1 of the survey are provided in Appendix A - Step 1 web questionnaire (PDF) and Appendix B - Step 1 paper questionnaires (PDF) of this report. The online and postal questionnaires for step 2 are provided in Appendix D - Step 2 web questionnaire (PDF) and Appendix E - Step 2 paper questionnaires (PDF) of this report.
Analysis of some of the previous questions will be presented in short topic reports, to be published in summer 2023.
Mailing strategy
The following overall participant engagement strategy was used, each item was sent to selected addresses in the post:
- invitation letter with the URL (Uniform Resource Locator) and sets of login details (two or four according to the assigned experimental condition) needed to access the survey online. Step 2 letters also contained Quick Response (QR) codes as an alternative method of accessing the survey. A Welsh version of the letter was also sent to addresses in Wales
- first reminder letter
- second reminder letter with two postal questionnaires9 and return envelopes
- third reminder letter.
The invitation letter and reminders, provided in Appendix C - Step 1 invitation and reminder letters (PDF) and Appendix F - Step 2 invitation and reminder letters (PDF), were the main levers to convince people to take part. The documents used in the pilot stage were reviewed and updated with the relevant participant selection criteria and also, the wording was strengthened to emphasise that those who did not gamble were invited to complete the survey.
All were carefully designed following the latest best practice and following the participant engagement guidance for online surveys published by the Office for National Statistics (ONS) (opens in new tab), drawing on their extensive testing in this area.
Experience shows that most people complete a survey within few days of receiving the request. The time between each mailing was therefore kept as short as possible, to ensure that the request was fresh in people’s mind. A gap of around 10 days between mailings was introduced, to allow removal of responding participants from the sample for the reminders. The day of the week of the mailing was varied to allow for the fact that different people may have time for survey participation on different days of the week.
A study website, freephone number and dedicated email address were set up for participants to contact with issues or queries. A £10 pounds completion incentive per individual questionnaire was offered. All online responders were emailed a 'Love2Shop' voucher code and postal responders were posted a voucher10.
Data preparation and checks
As described in previous sections, data was collected from two sources: an online questionnaire and a postal questionnaire. The online questionnaire included built-in routing and checks, whereas the postal questionnaire relied on correct navigation by participants and there was no constraint on the answers they could give.
The online questionnaire data in its raw form were available immediately to the research team. However, the postal questionnaire data had to be manually recorded as part of a separate process.
A number of rigorous quality assurance processes were utilised when preparing the survey data. These included checks that variables from the two data collection modes had merged correctly into one dataset. As up to four adults per household could answer demographic questions relating to the whole household (for example, household size and information about income), there was potential for differing responses between individuals.
The following rules for harmonising household responses were followed, in priority order:
- taking the most common valid answer (such as excluding 'don’t know', 'refusal')
- taking the valid answer from the oldest household member: or where this was not clear, the response of the first household member to complete a questionnaire (online completions first then postal completions).
A further step involved identifying and removing duplicate responses. For this, questionnaires were checked to see if responses to up to two or up to four questionnaires (depending on the experimental condition) were very likely to be from the same individual in a household (based on exact matches for the age, sex and name provided). Suspected duplicates were removed so that only one completed questionnaire from that individual was retained.
Where a household had more than the maximum number of records (two or four depending on the experimental condition), any extra cases were removed according to the following rules:
- fully completed online questionnaires took priority over postal questionnaires
- fully completed postal questionnaires took priority over partially completed online questionnaires
- partially completed online questionnaires took priority over partially completed postal questionnaires
- identifying and removing ‘speeders’ (individuals who completed the online questionnaire in an unrealistic amount of time for them to have properly engaged with the questions)11.
The data were then weighted to allow for comparisons with other data sources. The weighting strategy is outlined in Appendix G - Weighting technical summary (PDF).
References
8 How survey mode affects estimates of the prevalence of gambling harm: a multisurvey study (opens in new tab), Public Health, 204, 63-69, P. Sturgis and J. Kuha (2022).
9 This included C2 addresses where up to four adults were asked to participate.
10 'Love2Shop' vouchers cannot be exchanged for cash and cannot be used for gambling, so do not pose ethical problems for this survey.
11 'Speeders' were identified by calculating the median time it took to answer each question among all those who answered. From this an expected time was calculated for each participant dependent on the questions that they answered. A ratio of actual time compared with expected time was produced and any statistical outliers on this ratio measure were removed.
Survey design Next section
Response to the survey
Last updated: 18 April 2023
Show updates to this content
No changes to show.