Report
Gambling Survey for Great Britain - technical report
Gambling Survey for Great Britain - technical report
Methodology
Sampling strategy
A high-quality sample is essential for meeting the Gambling Commission’s (the Commission's) aims of creating a nationally representative new survey, capable of producing robust population estimates. To achieve this, a stratified random probability sample of addresses in Great Britain (GB) was used. The target population of the survey was adults aged 18 years and over, living in private households within GB.
There is no publicly available list of the GB adult population that can be used for sampling individuals. Instead, like many national surveys, the Postcode Address File (PAF) was used. The PAF is compiled by the Post Office and lists postal addresses (or postcode delivery points) in the United Kingdom.
To get from a list of addresses to a selection of adults within them, involves a two-stage selection process:
- selection of addresses from the PAF
- selection of adults within addresses.
Selection of addresses from the PAF
Prior to selection, the sample frame was ordered: this can help to reduce sampling error and increase the precision of estimates, as well as ensuring representativeness with respect to the measures used. The following measures for stratification (in order) were: Country and English region; Population density at Local Authority level and overall Index of Multiple Deprivation (IMD) score2.
At each sampled address, there may have been more than 1 dwelling and/or household. However, a random selection of households is very difficult to operationalise without an interviewer and there was no control over which household opened the invitation letter. As a result, in multi-occupied addresses, no formal household selection took place and the selection of which household took part was left to chance (that is, whichever household opened the letter). The overall proportion of multi-occupied addresses for PAF samples is very small (around 1 percent), and it is therefore unlikely to lead to any systematic error (known as bias) in the responding sample.
Selection of adults within addresses
At each address, up to 2 adults (aged 18 years and over) could take part. If the household contained 3 or more adults, the instruction was that the 2 adults with the most recent birthday should complete questionnaires.
Asking a set number of adults (in the case of this survey, 2) rather than all adults from each address to complete the survey is a well-established approach for push-to-web surveys in GB3. Most residential addresses (85 percent) contain either 1 or 2 adults, meaning that exclusion of additional adults should not introduce any notable error (known as selection bias). Under this approach, it is estimated that 93 percent of the sample are the ones that would have been selected using a random approach.
While this approach leads to a degree of within-household clustering, the effect of this is lower than if all adults per household were eligible though will be higher than if just 1 adult per household was selected. Moreover, the slight inefficiency at this stage is outweighed by the higher number of productive cases achieved from asking up to 2 adults from each address to complete the survey instead of only 1.
Mailing strategy
As a push-to-web survey using a PAF sample, the GSGB is reliant on sending invitation letters to perspective participants. The following participant engagement strategy was used; each item was sent to selected addresses in the post:
- invitation letter including a survey-specific web page and 2 sets of login details needed to access the online questionnaire.
- first reminder letter (this contained similar information to the initial invitation letter)
- second reminder letter with two postal questionnaires and return envelopes
- third reminder letter.
The letter also contained 2 Quick Response (QR) codes which provided an alternative method for accessing the online questionnaire. Instructions on what to do if more than 2 adults lived in the household were also included in the letter. Addresses in Wales received the letter in both Welsh and English.
The invitation letter and reminders were the main levers to convince people, including those who did not gamble, to take part. All were carefully designed following evidence-informed participant engagement guidance for online surveys published by the Office for National Statistics (ONS) Participant engagement for push-to-web social surveys – Government Analysis Function (opens in a new Tab).
Experience shows that most people complete a survey within a few days of receiving the request. The time between each mailing was therefore kept as short as possible, to ensure that the request was fresh in people’s mind. A gap of around 10 days between mailings was introduced, to allow removal of responding participants from the sample for the reminders. The day of the week of the mailing was varied to allow for the fact that different people may have time for survey participation on different days of the week.
A study website, freephone number and dedicated email address were set up for participants to contact with issues or queries. The use of monetary incentives in surveys has been proven to increase response rates4. A £10 completion incentive per individual questionnaire was offered. This took the form of a Love2Shop voucher. Those who responded online were emailed a Love2Shop voucher code. Those who completed the postal questionnaire received a physical Love2Shop voucher by post5.
Data collection
The aim was to achieve a sample size of 5,000 completed individual questionnaires per wave. To ensure a spread of completions throughout the data collection period, the sample for each wave was divided into 2 batches and issued at equal intervals (with minimal overlap between batches and waves).
Fieldwork dates for waves 1 and 2 were as follows (the first date for each wave refers to when invitation letters were posted; the latter date refers to the final date returned postal questionnaires were accepted):
- wave 1 - 31 July 2023 to 16 November 2023
- wave 2 - 6 November 2023 to 7 March 2024.
Questionnaire content and design
The postal questionnaire was designed to be as comparable as possible to the online questionnaire. This approach was taken to minimise the low risk of differences arising in the visual presentation of the 2 questionnaires, which could lead to differences in the ways in which questions were understood and answered (known as measurement differences).
Some differences between the 2 questionnaires remained. The online questionnaire included complex routing and dynamic adjustment of question wording that reflected the participant’s answers to earlier questions. This could not be replicated in the postal questionnaire. Moreover, to design a postal questionnaire that participants would find straightforward to complete within the required page limit, some questions asked in the online questionnaire were omitted from the postal version.
The questionnaires contained core and modular content. The core content was asked every wave and included some of the official statistics measures. Modular questions are asked on a rotating basis as required by the Commission and include topical questions or those related to the development of specific policies.
Core content included:
- leisure activities, internet access and use
- gambling activities participated in in-person and online in the last 12 months and in the past 4 weeks
- Problem Gambling Severity Index (PGSI)6
- own gambling harms and harms from others’ gambling7
- reasons for gambling
- how gambling makes participants feel
- health and wellbeing, including general health, smoking and drinking status, impulsivity scale, Short Warwick-Edinburgh Mental Well-Being Scale (SWEMWBS) and suicidality questions.
Modular content covers, but is not limited to:
- gambling management tools and complaints
- illegal gambling
- fairness and trustworthiness of gambling
- typologies (online participants only)
- gambling binge (online participants only)
- advertising and social media
- attitudes towards gambling (ATGS-8).
Demographic information captured:
- sex
- gender identity
- date of birth
- age
- ethnicity
- number of adults and children in the household
- marital or registered civil partnership status
- household income
- tenure
- education level
- economic activity8.
Data processing
Data was collected from 2 sources: an online questionnaire and a postal questionnaire. The online questionnaire included built-in routing and checks, whereas the postal questionnaire relied on correct navigation by participants and there was no constraint on the answers they could give. The online questionnaire data in its raw form were available immediately to the research team. However, the postal questionnaire data had to be manually recorded as part of a separate process.
A number of rigorous quality assurance processes were utilised when preparing the survey data. These included checks that variables from the 2 data collection modes had merged correctly into 1 dataset. As up to 2 adults per household could answer demographic questions relating to the whole household (for example, household size and information about income), there was potential for differing responses between individuals. The following rules for harmonising household responses were followed, in priority order:
- taking the most common valid answer (such as excluding ‘don’t know’, refusal)
- taking the valid answer from the oldest household member: or where this was not clear, the response of the first household member to complete a questionnaire (online completions first then postal completions).
A further step involved identifying and removing duplicate responses. For this, questionnaires were checked to see if responses to up to 2 questionnaires were very likely to be from the same individual in a household (based on exact matches for the age, sex and name provided). Suspected duplicates were removed so that only 1 completed questionnaire from that individual was retained.
Where a household had more than 2 records, any extra cases were removed according to the following rules:
- fully completed online questionnaires took priority over postal questionnaires
- fully completed postal questionnaires took priority over partially completed online questionnaires
- partially completed online questionnaires took priority over partially completed postal questionnaires.
‘Speeders’ (individuals who completed the online questionnaire in an unrealistic amount of time for them to have properly engaged with the questions) were identified and removed from the dataset9.
The data were then weighted to allow for comparisons with other data sources. Each publication is accompanied by a wave specific technical report, which outlines the weighting strategy used. Further details can be found alongside each publication in Latest Gambling Survey for Great Britain publications. Data will be deposited at UK Data Service (opens in a new Tab) after each annual publication.
References
2Indices of multiple deprivation (IMD) is a measure of relative deprivation for small, fixed geographic areas of the UK. Separate indices are produced for each UK country. IMD classifies these areas into 5 quintiles based on relative disadvantage, with quintile 1 being the most deprived and quintile 5 being the least deprived.
3In the Experimental Phase, the effect on data quality and selection bias of inviting a maximum of 2 or a maximum of 4 adults from each household to take part in the survey was investigated. There was no discernible experimental condition effect on household response rates, duplications nor gambling participation rates. There was evidence of significant clustering of gambling behaviours among households with three or four participants. As this can impact on the accuracy of the gambling participation data the recommendation was to invite up to two adults per household to take part going forward.
4See for example Church, A. (1993). Estimating the effect of incentives on mail survey response rates: A meta-analysis. Public Opinion Quarterly; 57:62-79. Mercer, A., Caporaso, A., Cantor, D. and Townsend, R. (2015). How Much Gets You How Much? Monetary Incentives and Response Rates in Household Surveys. Public Opinion Quarterly 79 (1):105-29. Pengli Jia, Luis Furuya-Kanamori, Zong-Shi Qin, Peng-Yan Jia, Chang Xu, Association between response rates and monetary incentives in sample study: a systematic review and meta-analysis, Postgraduate Medical Journal, Volume 97, Issue 1150, August 2021, Pages 501–510
5 Love2Shop vouchers cannot be exchanged for cash and cannot be used for gambling, so do not pose ethical problems for this survey
6The PGSI consists of 9 items and each item is assessed on a four-point scale: never, sometimes, most of the time, almost always. Responses to each item are given the following scores: never = 0, sometimes = 1, most of the time = 2, almost always = 3. When scores to each item are added up, a total score ranging from 0 to 27 is possible. A PGSI score of 8 or more represents a problem gambler. See Problem gambling screens (gamblingcommission.gov.uk) for full detail.
7The Commission is conducting work to develop and test a series of survey questions aimed at collecting data on the experience of gambling harms. Full detail on the work undertaken to date and the next steps can be found at Statistics and research series (gamblingcommission.gov.uk)
8Demographic questions align to the GSS harmonisation strategy which promotes consistent definitions and question wording in data collection.
9Speeders are identified by calculating the median time it took to answer each question among all those who answered. From this an expected time is calculated for each participant dependent on the questions that they answered. A ratio of actual time compared with expected time is produced and any statistical outliers on this ratio measure are removed.
GSGB 2024 technical report - Introduction Next section
GSGB 2024 technical report - Data analysis and reporting
Last updated: 1 March 2024
Show updates to this content
Formatting changes