This box is not visible in the printed version.
Gambling Survey for Great Britain - technical report
Published: 29 February 2024
Last updated: 29 February 2024
This version was printed or saved on: 20 May 2024
Online version: https://www.gamblingcommission.gov.uk/report/gambling-survey-for-great-britain-technical-report
This technical report provides detail on the background and methodology for Year 1 of the Gambling Survey for Great Britain (GSGB). Detail on the issued sample size, response and weighting strategy for each wave is provided in the wave-specific report. The wave-specific reports also provide detail of the online and postal questionnaires used in that wave.
Year 1 of the GSGB follows a period of development which has consisted of a pilot and experimental stage. Findings from the pilot were published in May 2022 and are reported in Participation and Prevalence: Pilot methodology review report. Two reports on findings from the experimental phase have been published. The first report, covering the first two steps, was published in April 2023 Gambling participation and the prevalence of problem gambling survey: Experimental statistics stage report. A further report, covering the final step, was published in November 2023 Gambling participation and the prevalence of problem gambling survey: Final experimental statistics stage (Step 3). Following the completion of the experimental phase, the survey moved to continuous official statistics data collection in July 2023.
The aims of the GSGB are to:
The GSGB uses what is known as a push-to-web approach, in which people are first encouraged to take part online, completing a web questionnaire. Those who do not initially take part online are subsequently offered an alternative means of participation. In the GSGB this alternative was a paper questionnaire, sent by post. By offering an alternative, the survey can include people who are not online or who do not feel willing or able to go online to take part. This helps improve the representativeness of the survey. Moreover, some gambling behaviours, notably the propensity to gamble online, are correlated with the probability to take part in online surveys, which can bias results1.
Inviting people to take part in the GSGB involved randomly selecting addresses within Great Britain, known as random probability sampling. This approach is discussed further in the next section.
1Sturgis, P., & Kuha, J. (2022). How survey mode affects estimates of the prevalence of gambling harm: a multisurvey study. Public Health, 204, 63-69.
A high-quality sample is essential for meeting the Gambling Commission’s (the Commission's) aims of creating a nationally representative new survey, capable of producing robust population estimates. To achieve this, a stratified random probability sample of addresses in Great Britain (GB) was used. The target population of the survey was adults aged 18 years and over, living in private households within GB.
There is no publicly available list of the GB adult population that can be used for sampling individuals. Instead, like many national surveys, the Postcode Address File (PAF) was used. The PAF is compiled by the Post Office and lists postal addresses (or postcode delivery points) in the United Kingdom.
To get from a list of addresses to a selection of adults within them, involves a two-stage selection process:
Prior to selection, the sample frame was ordered: this can help to reduce sampling error and increase the precision of estimates, as well as ensuring representativeness with respect to the measures used. The following measures for stratification (in order) were: Country and English region; Population density at Local Authority level and overall Index of Multiple Deprivation (IMD) score2.
At each sampled address, there may have been more than 1 dwelling and/or household. However, a random selection of households is very difficult to operationalise without an interviewer and there was no control over which household opened the invitation letter. As a result, in multi-occupied addresses, no formal household selection took place and the selection of which household took part was left to chance (that is, whichever household opened the letter). The overall proportion of multi-occupied addresses for PAF samples is very small (around 1 percent), and it is therefore unlikely to lead to any systematic error (known as bias) in the responding sample.
At each address, up to 2 adults (aged 18 years and over) could take part. If the household contained 3 or more adults, the instruction was that the 2 adults with the most recent birthday should complete questionnaires.
Asking a set number of adults (in the case of this survey, 2) rather than all adults from each address to complete the survey is a well-established approach for push-to-web surveys in GB3. Most residential addresses (85 percent) contain either 1 or 2 adults, meaning that exclusion of additional adults should not introduce any notable error (known as selection bias). Under this approach, it is estimated that 93 percent of the sample are the ones that would have been selected using a random approach.
While this approach leads to a degree of within-household clustering, the effect of this is lower than if all adults per household were eligible though will be higher than if just 1 adult per household was selected. Moreover, the slight inefficiency at this stage is outweighed by the higher number of productive cases achieved from asking up to 2 adults from each address to complete the survey instead of only 1.
As a push-to-web survey using a PAF sample, the GSGB is reliant on sending invitation letters to perspective participants. The following participant engagement strategy was used; each item was sent to selected addresses in the post:
The letter also contained 2 Quick Response (QR) codes which provided an alternative method for accessing the online questionnaire. Instructions on what to do if more than 2 adults lived in the household were also included in the letter. Addresses in Wales received the letter in both Welsh and English.
The invitation letter and reminders were the main levers to convince people, including those who did not gamble, to take part. All were carefully designed following evidence-informed participant engagement guidance for online surveys published by the Office for National Statistics (ONS) Participant engagement for push-to-web social surveys – Government Analysis Function (opens in a new Tab).
Experience shows that most people complete a survey within a few days of receiving the request. The time between each mailing was therefore kept as short as possible, to ensure that the request was fresh in people’s mind. A gap of around 10 days between mailings was introduced, to allow removal of responding participants from the sample for the reminders. The day of the week of the mailing was varied to allow for the fact that different people may have time for survey participation on different days of the week.
A study website, freephone number and dedicated email address were set up for participants to contact with issues or queries. The use of monetary incentives in surveys has been proven to increase response rates4. A £10 completion incentive per individual questionnaire was offered. This took the form of a Love2Shop voucher. Those who responded online were emailed a Love2Shop voucher code. Those who completed the postal questionnaire received a physical Love2Shop voucher by post5.
The aim was to achieve a sample size of 5,000 completed individual questionnaires per wave. To ensure a spread of completions throughout the data collection period, the sample for each wave was divided into 2 batches and issued at equal intervals (with minimal overlap between batches and waves).
Fieldwork dates for waves 1 and 2 were as follows (the first date for each wave refers to when invitation letters were posted; the latter date refers to the final date returned postal questionnaires were accepted):
The postal questionnaire was designed to be as comparable as possible to the online questionnaire. This approach was taken to minimise the low risk of differences arising in the visual presentation of the 2 questionnaires, which could lead to differences in the ways in which questions were understood and answered (known as measurement differences).
Some differences between the 2 questionnaires remained. The online questionnaire included complex routing and dynamic adjustment of question wording that reflected the participant’s answers to earlier questions. This could not be replicated in the postal questionnaire. Moreover, to design a postal questionnaire that participants would find straightforward to complete within the required page limit, some questions asked in the online questionnaire were omitted from the postal version.
The questionnaires contained core and modular content. The core content was asked every wave and included some of the official statistics measures. Modular questions are asked on a rotating basis as required by the Commission and include topical questions or those related to the development of specific policies.
Core content included:
Modular content covers, but is not limited to:
Demographic information captured:
Data was collected from 2 sources: an online questionnaire and a postal questionnaire. The online questionnaire included built-in routing and checks, whereas the postal questionnaire relied on correct navigation by participants and there was no constraint on the answers they could give. The online questionnaire data in its raw form were available immediately to the research team. However, the postal questionnaire data had to be manually recorded as part of a separate process.
A number of rigorous quality assurance processes were utilised when preparing the survey data. These included checks that variables from the 2 data collection modes had merged correctly into 1 dataset. As up to 2 adults per household could answer demographic questions relating to the whole household (for example, household size and information about income), there was potential for differing responses between individuals. The following rules for harmonising household responses were followed, in priority order:
A further step involved identifying and removing duplicate responses. For this, questionnaires were checked to see if responses to up to 2 questionnaires were very likely to be from the same individual in a household (based on exact matches for the age, sex and name provided). Suspected duplicates were removed so that only 1 completed questionnaire from that individual was retained.
Where a household had more than 2 records, any extra cases were removed according to the following rules:
‘Speeders’ (individuals who completed the online questionnaire in an unrealistic amount of time for them to have properly engaged with the questions) were identified and removed from the dataset9.
The data were then weighted to allow for comparisons with other data sources. Each publication is accompanied by a wave specific technical report, which outlines the weighting strategy used. Further details can be found alongside each publication in Latest Gambling Survey for Great Britain publications. Data will be deposited at UK Data Service (opens in a new Tab) after each annual publication.
2Indices of multiple deprivation (IMD) is a measure of relative deprivation for small, fixed geographic areas of the UK. Separate indices are produced for each UK country. IMD classifies these areas into 5 quintiles based on relative disadvantage, with quintile 1 being the most deprived and quintile 5 being the least deprived.
3In the Experimental Phase, the effect on data quality and selection bias of inviting a maximum of 2 or a maximum of 4 adults from each household to take part in the survey was investigated. There was no discernible experimental condition effect on household response rates, duplications nor gambling participation rates. There was evidence of significant clustering of gambling behaviours among households with three or four participants. As this can impact on the accuracy of the gambling participation data the recommendation was to invite up to two adults per household to take part going forward.
4See for example Church, A. (1993). Estimating the effect of incentives on mail survey response rates: A meta-analysis. Public Opinion Quarterly; 57:62-79. Mercer, A., Caporaso, A., Cantor, D. and Townsend, R. (2015). How Much Gets You How Much? Monetary Incentives and Response Rates in Household Surveys. Public Opinion Quarterly 79 (1):105-29. Pengli Jia, Luis Furuya-Kanamori, Zong-Shi Qin, Peng-Yan Jia, Chang Xu, Association between response rates and monetary incentives in sample study: a systematic review and meta-analysis, Postgraduate Medical Journal, Volume 97, Issue 1150, August 2021, Pages 501–510
5 Love2Shop vouchers cannot be exchanged for cash and cannot be used for gambling, so do not pose ethical problems for this survey
6The PGSI consists of 9 items and each item is assessed on a four-point scale: never, sometimes, most of the time, almost always. Responses to each item are given the following scores: never = 0, sometimes = 1, most of the time = 2, almost always = 3. When scores to each item are added up, a total score ranging from 0 to 27 is possible. A PGSI score of 8 or more represents a problem gambler. See Problem gambling screens (gamblingcommission.gov.uk) for full detail.
7The Commission is conducting work to develop and test a series of survey questions aimed at collecting data on the experience of gambling harms. Full detail on the work undertaken to date and the next steps can be found at Statistics and research series (gamblingcommission.gov.uk)
8Demographic questions align to the GSS harmonisation strategy which promotes consistent definitions and question wording in data collection.
9Speeders are identified by calculating the median time it took to answer each question among all those who answered. From this an expected time is calculated for each participant dependent on the questions that they answered. A ratio of actual time compared with expected time is produced and any statistical outliers on this ratio measure are removed.
The Gambling Survey for Great Britain (GSGB), in common with other surveys, collects information from a sample of the population. The sample is designed to represent the whole population of adults aged 18 years and over living in private households in Great Britain, as accurately as possible within practical constraints, such as time and cost. Consequently, statistics based on the survey are estimates, rather than precise figures, and are subject to a margin of error, also known as a 95 percent confidence interval. For example, the survey estimate might be 15 percent with a 95 percent confidence interval of 13 percent to 17 percent. A different sample might have given a different estimate, but it would be expected that the true value of the statistic in the population would be within the range given by the 95 percent confidence interval in 95 cases out of 100. Confidence intervals are affected by the size of the sample on which the estimate is based. Generally, the larger the sample, the smaller the confidence interval, which results in a more precise the estimate.
Confidence intervals are quoted for key statistics within GSGB reports. Where differences are commented on, these reflect the same degree of certainty that these differences are real, and not just within the margins of sampling error. These differences can be described as statistically significant10.
When commissioning the GSGB, the Gambling Commission (the Commission) weighed a range of strengths and limitations of the new approach. These are summarised in this section.
The Commission’s information needs are consolidated into a single survey (rather than several surveys as previously) which ensures consistency and efficiency.
The collection of data on a rolling basis and producing annual datasets and trends reduces the impact seasonal events (such as the FIFA World Cup) may have on key variables (for example, gambling participation rates).
The survey has undergone a comprehensive development stage, led by experts in the field. Development included cognitive testing, pilot testing, experimental testing, and stakeholder engagement.
The survey design and large, representative samples (per wave) allow the Commission to report on key results on a quarterly basis as well as to conduct more detailed analyses.
The push-to-web methodology is more cost effective when compared with face-to-face collection methods.
The methodology also allows increased numbers of people to be interviewed at relatively lower cost, something that is important for the analysis of gambling harms.
A postal alternative to the online questionnaire enabled the recruitment of adults who may have been less technologically literate, not have access to the Internet, or preferred an alternative option, and so increasing the representativeness of the sample.
The survey includes a broad range of content on gambling that is relevant to both gamblers and non-gamblers.
The self-administered data collection methods are likely to mitigate social desirability in responses to questions about sensitive topics (for example, about their gambling behaviour).
As the survey is ‘gambling focused’, it means more detail can be collected about gambling behaviours than is possible in a more general survey, where the number of questions that can be included is limited.
With a push-to-web methodology, interviewers are not present to collect the data in person and accuracy of answers relies on participants understanding the questions asked and following the instructions.
Similarly, there is a risk that some participants (although a small proportion) will not follow the routing instructions correctly on the postal version of the questionnaire. To minimise the risk, the postal questionnaire was designed with simple routing instructions and further, routing errors were checked and corrected during the office-based data editing process.
Compared with face-to-face interviewing methods, remote data collection methods typically have lower response rates, meaning they are potentially more susceptible to non-response bias. However, response rates for face-to-face interviews are also declining meaning these studies are also subject to non-response bias11. Furthermore, survey methodologists have found that the correlation between response rate and non-response bias is considerably weaker than conventionally assumed (Groves and Peytcheva 2008; Sturgis et al. 2017)12.
As the GSGB is ‘gambling focused’, it is possible that the survey disproportionately attracts those who gamble, so that this group may be over-represented.
The GSGB will produce new estimates of those scoring 1 to 2, 3 to 7 and 8 or higher on the Problem Gambling Severity Index (PGSI). No survey methodology is perfect; different surveys measuring the same phenomena will provide different estimates because variances in survey design and administration can affect both who takes part and how people answer these questions. Until 2010, data on gambling was captured through the bespoke British Gambling Prevalence Survey (BGPS) series (conducted in 1999, 2007 and 2010). Originally intended to be a tri-annual survey, funding for the BGPS was cut in 2011. The Commission then sought different ways to capture information about gambling within available budgets.
Between 2012 and 2021, the primary method of measuring scores according to the PGSI (as well as a second measurement instrument, the DSM-IV) was through the Health Survey for England (HSE series) and the Scottish Health Survey. The GSGB picks up where the BGPS left off by being a bespoke gambling survey that captures a wide range of information about gambling across the whole of Great Britain. However, the methodology for the new GSGB differs from the BGPS and the health survey series in a number of ways. The remainder of this section considers a range of issues affecting all surveys, that may either serve to under-estimate or over-estimate the PGSI estimates.
Using the PAF as a sample frame is common on large-scale surveys, including the BGPS, the GSGB and the health survey series. This means that only those living in private households are eligible to be included in the survey. People living in student halls of residence, military barracks, hospitals, prisons and other institutions are excluded. Some of these populations may have higher rates of gambling and higher PGSI scores. All studies using the PAF as a sample frame inherit this source of bias.
This bias in founded on the idea that there are social norms that govern certain behaviours and attitudes, and that people may misrepresent themselves so as to appear to conform to these norms13. In the survey context, this misrepresentation may involve participants explicitly deciding to give false information or modifying their in-mind answer. However, it can also involve participants giving information that they believe to be true but is in fact inaccurate14. It is a potential risk for all surveys that collect information on sensitive topics, including the health survey series and the GSGB. Sensitive topics include those that:
One strategy to reduce the risk of social desirability bias is to use self-completion methods. These methods include online and postal questionnaires, which are completed by the participant. Self-completion methods are used on both the health survey series and the GSGB to collect information on gambling. However, the surveys differ in the way in which self-completion methods are used, which may affect resulting estimates. The health survey series is an interviewer-administered survey that includes a paper self-completion questionnaire to ask about gambling behaviour. This is typically completed by participants in the presence of the interviewer and potentially other household members, who also take part in the survey.
Sturgis and Kuha15noted that it is possible that the presence of an interviewer or other household members might lead to underreporting of gambling in the self-completion questionnaire. Their analysis did not find a statistically significant difference in the proportion of people with a PGSI score of 1 or more within Health Survey for England (HSE) data, depending on whether other people were present at the time the gambling questions were being completed. However, subsequent analysis of HSE 2018 data conducted for the GSGB pilot, using multi-variate regression models, found that the odds of having an PGSI of 1 or more were 1.5 times higher among those who did not have other household members present at the point of interview16. The authors concluded that the online methods of GSGB may offer greater privacy to participants, and so reduce social desirability bias. A recent review of the GSGB development has suggested that additional research could be undertaken to better understand the role of socially desirable responding as the driver of the difference in gambling estimates between in-person and self-completion surveys17.
During the stakeholder engagement sessions conducted for the GSGB, those with lived experience of gambling harms stated that they would have been unlikely to participate in a survey when they were experiencing gambling difficulties. This was also highlighted by Sturgis in his review of the GSGB methods. Evidence supporting this is provided by analysis on non-response of the 2007 and 2010 BGPS series. In 2007, Scholes et al demonstrated a strong relationship between the factors predicting household non-response and gambling frequency: area and household-level factors which predicted lower household response were associated with higher gambling frequency. This suggests that those households less likely to take part in surveys were more likely to contain frequent gamblers18. Similar analysis conducted for the BGPS 2010 (reported in Wardle et al, 2014)19 demonstrated that households which either:
a) required multiple attempts to contact
b) were reissued after multiple follow-up attempts, or
c) were followed-up by telephone interviewers after the face-to-face interviewer had been unable to make contact were more likely to contain people who gambled.
This supports the notion that very engaged gamblers may be less likely to take part in surveys overall. This is likely to apply to all surveys. (However, both the health survey series and the GSGB are likely subject to different selection biases.)
The measurement of experience of so-called problem gambling is via a series of questions known as “screens”. Multiple different screens to measure the experience of problem gambling exist. No screen is perfect. In the BGPS and health survey series, two different screening instruments have been used: the DSM-IV and the PGSI Problem gambling screens.
Analysis of these screens shows that they capture different groups of people with potentially different types of problems. Orford et al suggested that the PGSI, especially among women, may underestimate certain forms of gambling harms which the DSM-IV is better suited to measure20. For this reason, the BGPS and health survey series always included both the DSM-IV and PGSI screens. The rates of problem gambling reported by the PGSI are lower than those reported by the DSM-IV. Since the BGPS was developed, the PGSI has become one of the most widely used screens, particularly because it presents scores on a spectrum of severity. In addition, there is now greater focus on the wider range of negative consequences associated with gambling which are not captured by the PGSI or the DSM-IV. During consultation on the GSGB questionnaire content, stakeholders strongly suggested that it would be appropriate to include only one screen for problem gambling and to use additional questionnaire space to capture other important aspects of gambling experiences. As a result, the GSGB only includes the PGSI screen, which generates lower estimates of problem gambling than the DSM-IV.
How surveys are presented to potential participants can influence who takes part. Williams and Volberg21 conducted an experiment presenting the same survey to potential participants but varying its description – introducing it either as a health and recreation survey or a gambling survey. The found that rates of problem gambling were higher in the latter. This is maybe because people who gamble may potentially be more likely to take part in a gambling survey because it is relevant to them. The GSGB likely suffers from this selection bias compared with the health survey series. Ethically, the GSGB invite letter has to inform people what the study is about which may make it more attractive to those who gamble. Despite best efforts to reduce this possibility, it is likely that some selection bias remains and so that rates of past-year gambling participation and PGSI scores are higher in the GSGB compared with the health survey series (when compared using the exact same questions, as was the case for the pilot and stage 1 of the experimental statistics phase).
In addition, analysis conducted by Sturgis and Kaha22 and also, Ashford and others (the latter for the GSGB pilot) detailed in the Participation and Prevalence: Pilot methodology review report found that those who completed PGSI questions online had higher PGSI scores than those who completed the questions via an alternative mode. In short, online surveys may overestimate the proportion of online gamblers, which may in turn overestimate gambling harm because online and frequent gambling are independently associated with a higher probability of gambling harm.
However, evidence suggests that those experiencing harms from gambling are less likely to take part in surveys overall and have poorer health outcomes. Given this, there is also the possibility that these people may be less likely to take part in a health-focused survey, which would also impact on the results obtained by health surveys. This is a theoretical possibility that needs further empirical examination, as recently recommended by Sturgis in his review of the GSGB development23.
10Statistical significance does not imply substantive importance; differences that are statistically significant are not necessarily meaningful or relevant.
11For example, in 2021, the Health Survey for England (HSE) household response rate was 32% compared with 60% in 2015 and 59% in 2018.
12Sturgis, Patrick, Joel Williams, Ian Brunton-Smith, and Jamie Moore. 2017. ‘Fieldwork Effort, Response Rate, and the Distribution of Survey Outcomes: A Multilevel Meta-Analysis’. Public Opinion Quarterly 81(2): 523–42; Groves, Robert M., and Emilia Peytcheva. 2008. ‘The Impact of Nonresponse Rates on Nonresponse Bias: A Meta-Analysis’. Public Opinion Quarterly 72(2): 167–89.
13Kreuter, F., Presser, S. and Tourangeau, R. (2008) ‘Social Desirability Bias in CATI, IVR, and Web Surveys: The Effects of Mode and Question Sensitivity’, Public Opinion Quarterly, 72(5), pp. 847–865.
14Tourangeau, R. and Yan, T. (2007) ‘Sensitive Questions in Surveys’, Psychological Bulletin, 133(5), pp. 859–883.
15Sturgis, P., & Kuha, J. (2022). How survey mode affects estimates of the prevalence of gambling harm: a multisurvey study. Public Health, 204, 63-69.
16Ashford, R., Bates, B., Bergli C et al (2022) Gambling participation and the prevalence of problem gambling survey: Pilot stage Methodology review report. National Centre for Social Research: London.
17Sturgis, Patrick (2024) Assessment of the Gambling Survey for Great Britain (GSGB). London School of Economics and Political Science, London, UK Assessment of the Gambling Survey for Great Britain (GSGB) - LSE Research Online (opens in new tab).
18Scholes, S., Wardle, H., Sproston, K., et al (2008) Understanding non-response to the British Gambling Prevalence Survey 2007. Technical Report. National Centre for Social Research, London.
19Wardle, H., Seabury, C., Ahmed, H et al (2014). Gambling Behaviour in England and Scotland. Findings from the Health Survey for England 2012 and the Scottish Health Survey 2012. National Centre for Social Research: London. Available at: Gambling behaviour in England and Scotland -Findings from the Health Survey for England 2012 and Scottish Health Survey 2012 (opens in new tab).
20Orford, J., Wardle, H., Griffiths, M., Sproston, M., Erens, B. (2010) PGSI and DSM-IV in the 2007 British Gambling Prevalence Survey: reliability, item response, factor structure and inter-scale agreement, International Gambling Studies, 10:1, 31-44.
21Williams, R. J., & Volberg, R. A. (2009). Impact of survey description, administration format, and exclusionary criteria on population prevalence rates of problem gambling. International Gambling Studies, 9(2), 101–117.
22Sturgis, P., & Kuha, J. (2022). How survey mode affects estimates of the prevalence of gambling harm: a multisurvey study. Public Health, 204, 63-69.
23Sturgis, Patrick (2024) Assessment of the Gambling Survey for Great Britain (GSGB). London School of Economics and Political Science, London, UK Assessment of the Gambling Survey for Great Britain (GSGB) - LSE Research Online (opens in new tab).