limitations of crime research

The accuracy of crime statistics: assessing the impact of police data bias on geographic crime analysis

• Open access
• Published: 26 March 2021
• Volume 18 , pages 515–541, ( 2022 )

Cite this article

You have full access to this open access article

• David Buil-Gil   ORCID: orcid.org/0000-0002-7549-6317 1 ,
• Angelo Moretti   ORCID: orcid.org/0000-0001-6543-9418 2 &
• Samuel H. Langton   ORCID: orcid.org/0000-0002-1322-1553 3  

26k Accesses

24 Citations

32 Altmetric

Explore all metrics

Police-recorded crimes are used by police forces to document community differences in crime and design spatially targeted strategies. Nevertheless, crimes known to police are affected by selection biases driven by underreporting. This paper presents a simulation study to analyze if crime statistics aggregated at small spatial scales are affected by larger bias than maps produced for larger geographies.

Based on parameters obtained from the UK Census, we simulate a synthetic population consistent with the characteristics of Manchester. Then, based on parameters derived from the Crime Survey for England and Wales, we simulate crimes suffered by individuals, and their likelihood to be known to police. This allows comparing the difference between all crimes and police-recorded incidents at different scales.

Measures of dispersion of the relative difference between all crimes and police-recorded crimes are larger when incidents are aggregated to small geographies. The percentage of crimes unknown to police varies widely across small areas, underestimating crime in certain places while overestimating it in others.

Conclusions

Micro-level crime analysis is affected by a larger risk of bias than crimes aggregated at larger scales. These results raise awareness about an important shortcoming of micro-level mapping, and further efforts are needed to improve crime estimates.

Similar content being viewed by others

The Impact of Measurement Error in Regression Models Using Police Recorded Crime Rates

Too Fine to be Good? Issues of Granularity, Uniformity and Error in Spatial Crime Analysis

More places than crimes: implications for evaluating the law of crime concentration at place.

Avoid common mistakes on your manuscript.

Introduction

Police-recorded crimes are the main source of information used by police forces and government agencies to analyze crime patterns, investigate the geographic concentration of crime, and design and evaluate spatially targeted policing strategies and crime prevention policies (Bowers and Johnson 2014 ; Weisburd and Lum 2005 ). Police statistics are also used by criminologists to develop theories of crime and deviance (Bruinsma and Johnson 2018 ). Nevertheless, crimes known to police are affected by selection biases driven by unequal crime reporting rates across social groups and geographical areas (Buil-Gil et al. 2021 ; Goudriaan et al. 2006 ; Hart and Rennison 2003 ; Xie 2014 ; Xie and Baumer 2019a ). The level of police control (e.g., police patrols, surveillance) also varies across areas, which may affect victims’ willingness to report crimes to police and dictate the likelihood that police officers witness incidents in some places more than others (McCandless et al. 2016 ; Schnebly 2008 ). The sources of measurement error that affect the bias and precision of crime statistics is an issue that merits scrutiny, since it affects policing practices, criminal justice policies, and citizens’ daily lives. Yet, it is an understudied issue.

The implications of crime data biases for documenting and explaining community differences in crime and guiding policing operational decision-making processes are mostly unknown (Brantingham 2018 ; Gibson and Kim 2008 ; Kirkpatrick 2017 ). Moreover, police analyses and crime mapping are moving toward using increasingly fine-grained geographic units of analysis, such as street segments and micro-places containing highly homogeneous communities (Groff et al. 2010 ; Weisburd et al. 2009 , 2012 ). Geographic crime analysis based on police-recorded crime and calls for service data is used to identify the micro-places where crime is most prevalent in order to effectively target police resource (Braga et al. 2018 ). In this context, we define “micro-places” as very detailed spatial units of analysis such as addresses, street segments, or clusters of such units (Weisburd et al. 2009 ). Despite the increasing interest in small units of analysis, the extent to which such aggregations impact on the overall accuracy of statistical outputs and spatial analyses remains unknown (Ramos et al. 2020 ). In other words, we do not know whether aggregating crime data at such detailed levels of analysis increases the impact of biases introduced by underreporting. This article presents a simulation study to analyze the impact of data biases on geographic crime analysis conducted at different spatial scales. The open question that this research aims to address is whether aggregating crimes at smaller, more socially homogeneous spatial scales increases the risk of obtaining biased outputs compared with aggregating crimes at larger, more socially heterogeneous geographical levels.

Since the early 1830s, numerous researchers have expressed concern about the limitations of using official statistics to analyze crime patterns across space and time (Kitsuse and Cicourel 1963 ; Skogan 1974 ). Soon after the publication of the first judiciary statistics in France, Alphonse de Candolle ( 1987a [1830], 1987b [1832]) cautioned that the validity of these data was likely to be affected by various sources of measurement error. For instance, crimes may not be discovered by victims, some victims may not report crimes to the authorities, offenders’ identities may remain unknown, and legal procedures may not lead to conviction. Moreover, cross-sectional comparisons of the number of people convicted in court are likely to be affected by changes in prosecution activity, and the proportion of recorded crimes to unknown offences may vary between countries (Aebi and Linde 2012 ). De Candolle ( 1987b [1832]) argued that the number of persons accused of crime was a better indicator of crime incidence than the number of persons convicted, since the former is closer to crime events in terms of legal procedure. This rationale was later used to describe the so-called “Sellin’s dictum” (i.e., “the value of a crime rate for index purposes decreases as the distance from the crime itself in terms of procedure increases,” Sellin 1931 : 346), and it is the main reason why crime incidents known to the police are generally preferred over judiciary statistics when it comes to analyzing crime. Police-recorded crimes, however, are also subject to criticism over the validity of recording and reporting. So much so that such data lost the official designation of National Statistics in the UK in 2014 (UK Statistics Authority 2014 ).

A key issue of concern regarding the use of police records for crime analysis and mapping is the fact that crime reporting rates are unequally distributed across social groups and geographic areas. Crime reporting to police forces is known to be more common among female victims than male victims, and young citizens report crimes less often than adults (Hart and Rennison 2003 ; Tarling and Morris 2010 ). There are also contextual factors that affect crime reporting rates across areas, such as neighborhood economic deprivation, the degree of urbanization, the concentration of minorities, and social cohesion (Berg et al. 2013 ; Goudriaan et al. 2006 ; Slocum et al. 2010 ; Xie and Baumer 2019a , b ; Xie and Lauritsen 2012 ). The demographic and social characteristics of small areas are generally more homogeneous compared with larger scales (e.g., Brattbakk 2014 ; Weisburd et al. 2012 ). Thus, crime aggregates produced at the level of small geographies are more likely to be affected by unequal crime reporting rates across groups compared with aggregates and maps produced at larger, more heterogeneous spatial scales. For instance, Buil-Gil et al. ( 2021 ) show that the variation in the “dark figure of crime” (i.e., all crimes not shown in police statistics) between neighborhoods (within cities) is larger than the variation between cities. We expect the risk of police data bias to be especially large when aggregating crime records at the level of micro-places.

This paper is organized as follows: sect. “ The criminology of place ” introduces the move toward low-level crime analysis in criminology. Section “Geographic crime analysis and measurement error ” discusses the various sources of measurement error that may affect police records and introduce bias into our understanding of community differences in crime. Section “ Data and methods ” introduces the data, methods, and steps taken to generate the synthetic population for our simulation study, and methods used to assess the findings. Section “ Mapping the bias of police-recorded crimes ” reports the results of the simulation study. Finally, sect. “ Discussion and conclusions ” discusses the findings and presents the conclusions and limitations, along with suggestions for future research.

The criminology of place

In the 1980s, several researchers began analyzing the concentration of crime in places and found that a large proportion of crimes known to the police concentrated in a small number of micro-places. Pierce et al. ( 1988 ) showed that 50% of all calls for police services in Boston took place in just 2.6% of addresses, suggesting that a disproportionately large volume of total crime could be attributed to just a handful of places. A year later, Sherman et al. ( 1989 ) conducted similar research in Minneapolis, obtaining almost the same results: 2.5% of addresses in this city generated 50% of all crime calls to the police. These were only two of the first studies looking into the concentration of crime in places. Since then, many other researchers have published remarkably similar findings (see a review in Lee et al. 2017 ). Environmental criminologists argue that the social and contextual conditions that favor crime vary across micro-places, and that opportunities for crime are structured within very small geographic areas (Brantingham and Brantingham 1995 ; Weisburd et al. 2012 ).

Given the persistency of this finding across multiple study sites and countries, Weisburd ( 2015 : 138) argues for a so-called “law of crime concentration” at micro-places, namely, that “for a defined measure of crime at a specific microgeographic unit, the concentration of crime will fall within a narrow bandwidth of percentages for a defined cumulative proportion of crime.” This has served as a basis for police forces all over the world to develop place-based strategies that increase police control over those areas where crime is highly concentrated to efficiently reduce citywide crime (Braga et al. 2018 ; Groff et al. 2010 ; Kirkpatrick 2017 ).

However, the vast majority of research analyzing crime concentration, and evaluating the impact of place-based policing interventions, is based on data about crimes known to the police. For instance, 41 out of 44 studies examining the crime concentration at places reviewed by Lee et al. ( 2017 ) used crime incidents reported to police, and 4 out of 44 analyzed calls for police services (note that some studies used more than one source of data). Both these sources of data depend on citizens’ willingness to report crimes and cooperate with the police, which are known to be affected by the social and demographic characteristics of individuals, but also by variables that operate at the scales of small communities, such as concentrated disadvantage, perceived disorder, and collective efficacy (Jackson et al. 2013 ). Weisburd et al. ( 2012 : 5) argue that “the criminology of place [...] emphasizes the importance of micro-units of geography as social systems relevant to the crime problem.” And yet, these micro-level social systems may also be key in explaining why crime reporting rates—and thus the likelihood of crimes being known to police—are high in some places and low in others, and as such, we might expect that the sources of measurement error that affect police data will vary across micro-places.

Geographic crime analysis and measurement error

There are four primary sources of data bias that may affect the accuracy of community differences in crime documented through police statistics. First, the willingness of residents to report crimes to police is known to be associated with individual and contextual factors that vary across geographic areas (Hart and Rennison 2003 ). There are demographic, social, economic, and environmental factors that affect crime reporting rates. For example, the victims’ sex, age, employment status, education level, and ethnic group are all good predictors of their likelihood to report crimes to the police (Hart and Rennison 2003 ). Since some of these resident characteristics concentrate in particular areas, we also expect crime reporting rates to vary across areas. Generally, deprived neighborhoods and areas with large concentrations of immigrants have lower crime reporting rates than middle-class areas (Baumer 2002 ; Xie and Baumer 2019a ; Goudriaan et al. 2006 ), and crimes that take place in cohesive areas have a higher chance of being known to the police (Goudriaan et al. 2006 ; Jackson et al. 2013 ). Moreover, residents from rural areas are generally more willing to cooperate with police services than urban citizens (Hart and Rennison 2003 ). Research has also found that the incident seriousness and harm are very strongly linked to the reporting decision (Baumer 2002 ; Xie and Baumer 2019b ).

Second, studies have found that the overall crime rate and citizens’ perceptions about police forces, which also vary across areas, affect residents’ willingness to cooperate with the police (e.g., Xie 2014 ). Berg et al. ( 2013 ) show that the most important contextual factor in explaining crime reporting is the level of crime in the area. Jackson et al. ( 2013 ) argue that the level of trust in police fairness and residents’ perceptions of police legitimacy is key to predict the willingness to cooperate with police forces.

Third, unequal police control across areas may inflate crime statistics in some places but not others. Schnebly ( 2008 ) shows that cities with more police officers trained in community-oriented policing generally have higher rates of police notification, whereas McCandless et al. ( 2016 ) argue that poorly handled stop and search practices may discourage residents from engaging with the police.

Fourth, there may be differences between counting rules applied by different police forces (Aebi and Linde 2012 ). This is not expected to be a major source of error in England and Wales, since all 43 police forces follow common counting rules (National Crime Recording Standards and Home Office Counting Rules for Recorded Crime). Nevertheless, we note that, in 2014, Her Majesty’s Inspectorate of Constabulary and Fire & Rescue Services conducted an inspection about police statistics and concluded that the extent to which certain counting practices was followed varied between police forces (HMIC 2014 ).

Some of these sources of measurement error were mentioned by Skogan ( 1977 : 41) to argue that the dark figure of crime “limits the deterrent capability of the criminal justice system, contributes to the misallocation of police resources, renders victims ineligible for public and private benefits, affects insurance costs, and helps shape the police role in society.” Moreover, the UK public administration also acknowledges that “there is accumulating evidence that suggests the underlying data on crimes recorded by the police may not be reliable” (UK Statistics Authority 2014 : 2). As a consequence, in 2014, crime data were removed from the UK National Statistics designation.

Given that many of the factors generating disparities in the bias and precision of police-recorded crime data are non-uniformly distributed across space, even in the same city, it is plausible that the bias affecting crime data varies considerably between small areas. Indeed, issues of bias and precision may even be compounded as the geographic resolution becomes more fine-grained. Oberwittler and Wikström ( 2009 : 41) argue that, in order to analyze crime, “smaller geographical units are more homogeneous, and hence more accurately measure environments. In other words, smaller is better.” Smaller units of analysis are said to be better for explaining criminal behaviors since crime is determined by opportunities that occur in the immediate environment. However, smaller units of analysis may also be preferred to explain the amount of crime which remains hidden in police statistics (either because victims and witnesses fail to report or because the police fail to record). The “aggregation bias,” which argues that what is true for a group should also be true for individuals within such a group, tends to be used to justify the selection of smaller spatial units in geographic crime analysis due to this homogeneity in residential characteristics. And yet, high internal homogeneity and between-unit heterogeneity may generate greater variability in bias and precision between units. It would be paradoxical and self-defeating if, in seeking to avoid aggregation bias with the use of micro-scale units, studies increase the risk of crime statistics being affected by bias and imprecision. This would have significant repercussions for academic endeavor and policing practices that document and explain community differences in crime.

Data and methods

Simulation studies are computer experiments in which data is created via pseudo-random sampling in order to evaluate the bias and variance of estimators, compare estimators, investigate the impact of sample sizes on estimators’ performance, and select optimal sample sizes, among others (Moretti 2020 ). Brantingham and Brantingham ( 2004 ) recommend the use of computer simulations to understand crime patterns and provide policy guidance for crime control (see also Groff and Mazerolle 2008 ; Townsley and Birks 2008 ). In this study, we generate a synthetic dataset of crimes known and unknown to police in Manchester, UK, and aggregate crimes at different spatial scales. This permits an investigation into whether aggregates of crimes known to police at the micro-scale level suffer from a higher risk of bias compared with those at larger aggregations, such as neighborhoods and wards.

Based on parameters obtained from the UK Census 2011 and Index of Multiple Deprivation (IMD) 2010, we simulate a synthetic individual-level population consistent with the characteristics of Manchester. The simulated population reflects the real distributions and parameters of variables related to individuals residing in each area of the city (i.e., mean, proportion, and variance of the citizens’ age, sex, employment status, education level, ethnicity, marriage status, and country of birth). The measure of multiple deprivation captures the overall level of poverty in each area. Then, based on parameters derived from the Crime Survey for England and Wales (CSEW) 2011/2012, we simulate the victimization of these individuals across social groups and areas and predict the likelihood of these crimes being known to the police. This allows us to compare the relative difference between all crimes and police-recorded incidents at the different spatial scales.

The main motivation for using a simulation study with synthetic data, instead of simply using crime records, is because the absolute number of crimes in places is an unknown figure, regardless which source of data we use (see sect. “ Geographic crime analysis and measurement error ”). Police records are affected by a diverse array of sources of error which vary between areas, and the CSEW sample is only designed to allow the production of reliable estimates at the level of police force areas (smaller areas are unplanned domains with very small sample sizes for which analyses based on direct estimates lead to unreliable outputs; Buil-Gil et al. 2021 ). Nevertheless, the analytical steps followed in this article are designed to provide an answer to our research question (namely, whether micro-level aggregates of police-recorded crime are affected by a larger risk of bias compared with larger scales), rather than producing unbiased estimates of crime in places. Future research will explore if the method used here is also a good way to produce accurate estimates of crime in places and compare these estimates with model-based estimates of crime indicators obtained from more traditional methods in small area estimation (Buil-Gil et al. 2021 ). Indeed, unbiased estimates of crime in places are needed to guide evidence-based policing and research.

In this section, we describe the data and methods used to generate the synthetic population of crimes known and unknown to police and evaluate differences between spatial scales. Section “ Generating the population and simulation steps ” outlines the data-generating mechanism and the steps of our simulation study, and in sect. “ Empirical evaluation of simulated dataset of crimes ,” we provide an empirical evaluation of the simulated dataset. We discuss methods to assess the results in sect. “ Assessing the results .”

Generating the population and simulation steps

The simulation of our synthetic population involves three steps which are described in detail below. All analyses have been programmed in R (R Core Team 2020 ), and all data and code used for this simulation study are available from a public GitHub repository (see https://github.com/davidbuilgil/crime_simulation2 ).

Step 1. Simulating a synthetic population from census data

The first step is to generate a synthetic population consistent with the social, demographic, and spatial characteristics of Manchester. We download aggregated data about residents at the output area (OA) level from the Nomis website ( https://www.nomisweb.co.uk/census/2011 ), which publishes data recorded by the UK Census 2011. For consistency, we will conduct all our analyses using information collected in 2011. From Nomis, we obtain census parameters of various variables in each OA in Manchester. OAs are the smallest geographic units for which census data are openly published in the UK. The minimum population size per OA is 40 households and 100 residents, but the average size is 125 households. We will also use other units of geography in further steps: lower layer super output areas (LSOAs), that generally contain between four and six OAs with an average population size of 1500; and middle layer super output areas (MSOAs), which have an average population size of 7200. The largest scale used are wards. In Manchester local authority, there are 1530 OAs, 282 LSOAs, 57 MSOAs, and 32 wards.

Although UK census data achieve nearly complete coverage of the population, and measurement error arising from using these data is likely to be very small, Census data are not problem-free. For instance, census non-response rates vary between age, sex, and ethnic groups (e.g., while more than 97% of females above 55 responded the census, the response rate for males aged 25 to 29 was 86%), and questionnaire items (e.g., non-response rates were 0.4% and 0.6% for sex and age questions, respectively, and 3%, 4%, and 5.7% for ethnicity, employment status, and qualifications questions). In Manchester, the census response rate was 89%. In order to adjust for non-response in census data, the Office for National Statistics used an edit and imputation system and coverage assessment and adjustment process before publishing data in Nomis (Compton et al. 2017 ; Office for National Statistics 2015 ). Census data are widely used as empirical values of demographic domains in areas for academic research and policy (Gale et al. 2017 ). From the census, we obtain the number of citizens living in each OA (i.e., resident population size), the mean and standard deviation of age by OA, and the proportion of citizens in each area with the following characteristics defined by binary variables (in parentheses, we detail the reference category): sex (male), ethnicity (white), employment status (population without any income), education (higher education or more), marriage status (married), and country of birth (born in the UK). We use this information to simulate our synthetic individual-level population and their corresponding social-demographic characteristics within each OA. Moreover, we attach the known IMD 2010 decile in each OA. This ensures that we account for both individual and area-level measures in our simulation. The IMD is a measure of multiple deprivation calculated by the UK Government from indicators of income, employment, health, education, barriers to housing and services, and crime and living environment at the small area level (McLennan et al. 2011 ). Generating these values allows us, in subsequent steps, to simulate crimes experienced by citizens, as well as the likelihood of each crime being known to the police, based on parameters obtained from survey data. We use these specific variables since these are known to be associated with crime victimization and crime reporting rates (see sect. “ Geographic crime analysis and measurement error ”). Thus, the selection of census parameters is driven by the literature review and the availability of data recorded by the census and IMD.

The variables are generated for d  = 1, …, D OAs and i  = 1, …, N d individual citizens according to the distributions detailed below, where N d denotes the population dimension in the d th OA:

\( \mathrm{Ag}{\mathrm{e}}_{di}\sim N\left({\mu}_d^{\mathrm{Age}},{\sigma}_d^{2,\mathrm{Age}}\right),\kern0.5em \) where \( {\mu}_d^{\mathrm{Age}} \) and \( {\sigma}_d^{2,\mathrm{Age}} \) denote the mean and variance of age for the d th OA.

\( \mathrm{Se}{\mathrm{x}}_{di}\sim \mathrm{Bernoulli}\left({\pi}_d^{\mathrm{Male}}\right) \) , where \( {\pi}_d^{\mathrm{Male}} \) denotes the proportion of males in d th OA.

\( {\mathrm{NoInc}}_{di}\sim \mathrm{Bernoulli}\left({\pi}_d^{\mathrm{NoInc}}\right) \) , where \( {\pi}_d^{\mathrm{NoInc}} \) denotes the proportion of citizens without any income in the d th OA.

\( {\mathrm{HE}}_{di}\sim \mathrm{Bernoulli}\left({\pi}_d^{\mathrm{HE}}\right) \) , where \( {\pi}_d^{\mathrm{HE}} \) denotes the proportion of citizens with high education (holding a university degree) in the d th OA.

\( \mathrm{Whit}{\mathrm{e}}_{di}\sim \mathrm{Bernoulli}\left({\pi}_d^{\mathrm{White}}\right) \) , where \( {\pi}_d^{\mathrm{White}} \) denotes the proportion of white citizens in the d th OA.

\( {\mathrm{Married}}_{di}\sim \mathrm{Bernoulli}\left({\pi}_d^{\mathrm{Married}}\right) \) , where \( {\pi}_d^{\mathrm{Married}} \) denotes the proportion of married population in the d th OA.

\( {\mathrm{BornUK}}_{di}\sim \mathrm{Bernoulli}\left({\pi}_d^{BornUK}\right) \) , where \( {\pi}_d^{\mathrm{BornUK}} \) denotes the proportion of population born in the UK in the d th OA.

Thus, we generate N = 503,127 units with their individual and contextual characteristics across D = 1,530 OAs in Manchester. Given that we simulate all individual information based on population parameters obtained from the census using small spatial units of analysis (i.e., OAs), our synthetic population is very similar (in terms of distributions and ranking) to the empirical population of each OA. The Spearman’s rank correlation coefficient of the mean of age, sex, income, higher education, ethnicity, marriage status, and country of birth across areas in census data and our simulated dataset is almost perfect (i.e., larger than 0.99 for all variables).

Step 2. Simulating crime victimization from CSEW data

We use parameters obtained from the CSEW 2011/2012 to generate the crimes experienced by each individual citizen. The CSEW is an annual victimization survey conducted in England and Wales. Its sampling design consists of a multistage stratified random sample by which a randomly selected adult (aged 16 or more) from a randomly selected household is asked about experienced victimization in the last 12 months (Office for National Statistics 2013 ). The survey also includes questions about crime reporting to the police and whether each crime took place in the local area, among others. The main part of the survey is completed face-to-face in respondents’ households, although some questions (about drugs and alcohol use, and domestic abuse) are administered via computer-assisted personal interviewing. The CSEW sample size in 2011/2012 was 46,031 respondents.

In order to simulate the number of crimes faced by each individual unit within our synthetic population of Manchester residents, we first estimate negative binomial regression models of crime victimization from CSEW data and then use the model parameter estimates to predict crime incidence within our simulated population. Given that different crime types are known to be associated with different social and contextual variables (Andresen and Linning 2012 ; Quick et al. 2018 ), and the variables associated with crime reporting to the police also vary according to crime type (Baumer 2002 ; Hart and Rennison 2003 ; Tarling and Morris 2010 ), we estimate one negative binomial regression model by each of four groups of crime types:

Vehicle crimes: includes the number of (a) thefts of motor vehicles, (b) things stolen off vehicles, and (c) vehicles tampered or damaged, all during the last 12 months.

Residence crimes: number of times (a) someone entered a residence without permission to steal, (b) someone entered a residence without permission to cause damage, (c) someone tried to enter a residence without permission to steal or cause damage, (d) anything got stolen from a residence, (e) anything stolen from outside a residence (garden, doorstep, garage), and (f) anything damaged outside a residence. These refer to events happening both at the current and previous households during the last 12 months.

Theft and property crimes (excluding burglary) : number of times (a) something stolen out of hands, pockets, bags, or cases; (b) someone tried to steal something out of hands, pockets, bags, or cases; (c) something stolen from a cloakroom, office, car or anywhere else; and (d) bicycle stolen, all during the last 12 months.

Violent crimes: number of times (a) someone deliberately hit the person with fists or weapon or used force or violence in any way, (b) someone threatened to damage or use violence on the person or things belonging to the person, (c) someone sexually assaulted or attacked the person, and (d) some member of the household hit or used weapon, or kicked, or used force in any way on the person, all during the last 12 months.

Thus, this approach assumes that distributions and slopes observed in the CSEW at a national level apply to crimes that take place in Manchester local authority. The CSEW sample for Manchester is not large enough to estimate accurate regression models, and thus, we use models estimated at a national level to estimate parameters used to generate crimes at a local level. The implications of taking this approach are further discussed in sect. “ Empirical evaluation of simulated dataset of crimes ”. To alleviate the concern about this potential limitation, we show in Appendix Table 7 that the negative binomial regression model of crime victimization estimated from respondents residing in urban and metropolitan areas (excluding London) shows very similar results to model results estimated from all respondents in England and Wales.

The negative binomial regression model is a widely adopted model in this context, which has been proven to adjust well to the skewness of crime count variables (Britt et al. 2018 ; Chaiken and Rolph 1981 ). To estimate the negative binomial regression models, we use the same independent variables described in step 1 (i.e., age, sex, employment status, education level, ethnic group, marriage status, country of birth, IMD decile). However, in this step, these are taken from the CSEW. This allows us to obtain the regression model coefficient estimates and dispersion parameter estimates (Table 1 ), denoted by \( {\hat{\ \alpha}}_p \) for a generic p independent variable and \( \hat{\ \theta } \) , respectively, that will be used to generate the crime counts per person in the synthetic population. Thus, regression models consider individual and area-level variables typically associated with crime victimization risk and crime reporting, but these do not account for other area-level contextual attributes associated with crime and crime reporting, such as the presence of crime generators and attractors in the area (Brantingham and Brantingham 1995 ). Since this is a new methodological approach, we include only a small number of variables recorded in the census and IMD to keep the model parsimonious, avoid multicollinearity, and improve the model accuracy. Models do not consider other important factors, such as individuals’ routine activities and alcohol consumption, because these are not recorded in the census.

Table 1 shows the negative binomial regression models used to estimate crime victimization from CSEW 2011/2012 data. Measures of pseudo- R 2 and normalized root mean squared error (NRMSE) indicate a good fit and accuracy of our models. We use the estimated regression coefficients to generate our synthetic population of crimes, but these also provide some information about which individual characteristics are associated with a higher or lower risk of victimization by crime type. For example, age is negatively associated with crime victimization in all crime types. Being male is a good predictor of suffering vehicle and property crimes, but not residence or violent crimes. With regards to income levels, those with some type of income have a higher risk of victimization by vehicle and violent crimes, whereas respondents without any income have a higher risk of suffering residence crimes. Citizens with a higher education degree generally suffer more property and vehicle crimes than residents without university qualifications, whereas those without higher education certificates are at a higher risk of suffering violent crimes. Married citizens tend to suffer more vehicle crimes, while non-married suffer more property and violent crimes. Citizens born in the UK experience more residence and vehicle crimes than immigrants. And areas with high values of deprivation concentrate more vehicle, residence, and property crimes.

Crime victimization counts for each unit in the simulated population are generated following a negative binomial regression model using the regression coefficient and dispersion parameter estimates obtained from the CSEW (Table 1 ) and the independent variables simulated in step 1. For example, we predict the number of vehicle crimes (Vehi i ) suffered by a given individual i as follows:

where NB denotes the negative binomial distribution, and:

We repeat this procedure for all four crime types. Thus, the variability and relationships between variables observed in the CSEW are reproduced in our simulated population, and we assume that these values represent the true extent of crime victimization in the population of Manchester. We evaluate the quality of the synthetic population of crimes in sect. “ Empirical evaluation of simulated dataset of crimes .”

Step 3. Simulating crimes known to police from CSEW data

The third step consists of estimating whether each simulated crime is known to the police or not. This allows us to analyze the difference between all crimes (generated in step 2), and those crimes known to the police (to be estimated in step 3) for each area in Manchester. First, we create a new dataset in which every crime generated in step 2 becomes the observational unit. Here, our units of analysis are crimes in places, instead of individual citizens; therefore, some residents may be represented more than once (i.e., those who suffered multiple forms of victimization).

In order to estimate the likelihood of each crime being known to the police, we follow a similar procedure as in step 2, but in this case, we make use of logistic regression models for binary outcomes, which are better described by the Bernoulli distribution of crime reporting. First, we estimate a logistic regression model of whether crimes are known to police or not. We use the CSEW dataset of crimes ( n  = 14,758), and fit the model using the same independent variables as in step 2 to estimate the likelihood of crimes being known to the police (see the results of logistic regression models in Table 2 ). We estimate one regression model per crime types to account for the fact that the crime type and incident seriousness are strongly linked to crime reporting (Baumer 2002 ; Xie and Baumer 2019b ). The CSEW asks each victim of each crime whether “Did the police come to know about the matter?” We use this measure to estimate our regression models. Thus, here, we estimate if the police knows about each crime, which is not always due to crime reporting (i.e., estimates from the CSEW 2011/2012 indicate that 32.2% of crimes known to the police were reported by another person, 2.3% were witnessed by the police and 2.2% were discovered by the police by another way).

Second, we estimate whether each crime in our simulated dataset is known to the police, following a Bernoulli distribution from the regression coefficient estimates shown in Table 2 and the independent variables simulated in step 1. As in the previous case, we repeat this procedure for each crime type, since some variables may affect some crime types in a different way than others (Xie and Baumer 2019a ). For example, to estimate whether each vehicle crime j , suffered by an individual i , is known to police (KVehi ji ), we calculate:

\( {\hat{\gamma}}_p \) denotes the regression model coefficient estimate for a p independent variable, and J denotes all simulated crimes. Measures of pseudo-R 2 show a good fit of models.

One important constraint of crime estimates produced from the CSEW is that these provide information about area victimization rates (i.e., number of crimes suffered by citizens living in one area, regardless of where crimes took place), instead of area offence rates (i.e., number of crimes taking place in each area). This may complicate efforts to compare and combine survey-based estimates with police records. Given that our simulated dataset of crimes is based on CSEW parameters and census data about residential population characteristics, our synthetic dataset of crimes is also likely to be affected by this limitation. In order to mitigate the impact of this shortcoming on any results drawn from our study, we follow similar steps as in step 3 in order to estimate whether each crime took place in the residents’ local area or somewhere else and remove from the study all those crimes that do not take place within 15-min walking distance from the citizens’ household (see Appendix 2). Our final sample size is 452,604 crimes distributed across 1530 OAs in Manchester. This facilitates efforts to compare our simulated dataset of crimes with police-recorded incidents, but we note that our synthetic dataset does not account for those crimes that take place in an area but are suffered by persons living in any other place. According to estimates drawn from the CSEW 2011/2012, this represents 26.0% of all crimes, which are likely to be overrepresented in commercial areas and business districts in the city center, where the difference between the workday population and the number of residents is generally very large (e.g., 490.2% in Manchester city center; Manchester City Council 2011 ). We return to this point in the discussion section to discuss ways in which this shortcoming may be further addressed in future research.

Empirical evaluation of simulated dataset of crimes

Once all synthetic data are generated, we use victimization data recorded by the CSEW and data about crimes known to Greater Manchester Police (GMP) to empirically evaluate whether our simulated dataset of crimes matches the empirical values of crime. This is used to evaluate the quality of our synthetically generated dataset of crimes.

First, Table 3 compares the average number of crimes suffered by individuals across socio-demographic groups as recorded by the CSEW 2011/2012 and our simulated dataset. The distribution of the synthetic dataset of crimes is very similar to that of the CSEW, but values appear to be slightly larger in the synthetic population than in the survey data. For instance, citizens younger than 35 suffer the most crimes in both datasets, and males suffer more vehicle, residence, and property crimes. Crime victimization differences by ethnicity, employment status, education level, marriage status, country of birth, and IMD decile shown in the CSEW are also observed in the simulated dataset of crimes. In the case of residence crimes, incidences in our simulated population appear to be slightly larger than those observed in the CSEW. We note that our simulated dataset refers to crimes taking place in Manchester local authority, whereas the CSEW reports data for all England and Wales. In 2011/2012, the overall rate of crimes known to police per 1000 citizens was notably larger in Manchester than in the rest of England and Wales (Office for National Statistics 2019 ), and the Crime Severity Score for 2011/2012 (an index that ranks the severity of crimes in each local authority) was 104.6% larger in Manchester than the average of England and Wales (Office for National Statistics 2020 ). Therefore, the differences observed between CSEW and our synthetic population of crimes are likely to reflect true variations between the crime levels in Manchester and England and Wales as a whole.

Second, Table 4 presents the proportion of crimes that are known to the police grouped by the socio-demographic and contextual characteristics of victims in CSEW and our simulated data. By looking at the table, we see that the proportions related to the CSEW are very similar to the ones obtained on the simulated data. This shows that modeling results are consistent, thus preserving relationships between variables.

Third, we download crime data recorded by GMP ( https://data.police.uk/ ) and compare area-level aggregates of crimes known to GMP with our synthetic dataset of crimes known to the police. To do this, we only consider those simulated crimes that were estimated as being known to police and taking place in the local area. Spearman’s rank correlation and Global Moran’s I coefficients between the area-level aggregates of our synthetic dataset of crimes and crimes known to GMP are reported in Table 5 . Tiefelsdorf’s ( 2000 ) exact approximation of the Global Moran’s I test is used as a measure of spatial dependency between the two measures, to analyze if the number of crimes in our simulated dataset is explained by the value of crimes known to GMP in surrounding areas (Bivand et al. 2009 ).

We aggregate all crimes known to police to each spatial unit using the “sf” package in R (Pebesma 2018 ). Out of the 87,457 crimes known to GMP, 642 could not be geocoded. We note that we obtained slightly different results using two different analytical approaches to aggregating crimes in areas (i.e., counting crimes in OAs and then aggregating from OAs to LSOA, MSOAs, and wards using a lookup table, versus counting crimes in OAs, LSOAs, MSOAs, and wards, respectively), which may be due to errors arising from the aggregation process or inconsistencies in the lookup table. We chose the second approach (i.e., counting points in polygons at the different scales), since, on average, a larger number of offences were registered in each area using this method. Tompson et al. ( 2015 ) demonstrate that open crime data published in England and Wales is spatially precise at the levels of LSOA and MSOA, but that the spatial noise added to these data for the purposes of anonymity means that OA-level maps often have inadequate precision. Thus, we only present and discuss the results obtained at LSOA and larger spatial levels.

Table 5 shows positive and statistically significant coefficients of Spearman’s rank correlation for all crime types at the LSOA level. The index of Global Moran’s I is also statistically significant and positive in all cases. At the MSOA and ward levels, the coefficients of Spearman’s correlation for vehicle crimes are not statistically significant. This is likely to be explained by the small number of MSOAs and wards under study (56 and 32, respectively). Generally speaking, our simulated dataset of synthetic crimes is a good indicator of crimes known to police, although both datasets are not perfectly aligned. Our synthetic dataset of crimes may underestimate crimes known to police in areas with a large difference between workday and residential populations, but it appears to be a precise indicator of crimes known to police in residential areas. In the discussion section, we present some thoughts about how to address this in future research.

Assessing the results

In order to assess the extent to which the number of simulated crimes known to police varies from all simulated crimes at the different spatial scales, we calculate the absolute percentage relative difference (RD) and the percentage relative bias (RB) between these two values for each crime type in each area at four spatial scales.

First, RD is calculated for every area d in the specified level of geography (i.e., Geo = {OA, LSOA, MSOA, wards}), as follows:

where E d denotes the count of all crimes in area d , and K d is the count of crimes known to police in the same area.

Second, RB is computed as follows:

We evaluate the average RD and RB at the different spatial scales, but also their spread, to establish if the measures of dispersion across areas become larger when the geographic scale becomes smaller. This permits a demonstration not just of the mean differences between all crimes and crimes known to police at different spatial scales but also the variability in these differences, to help shed light on whether there is higher variability at fine-grained spatial scales. This is investigated via the standard deviation (SD), minimum, maximum, and mean of the RD and RB at the different scales. In addition, boxplots and maps are shown to visualize outputs.

Mapping the bias of police-recorded crimes

This section presents the results of the simulation study. More specifically, we analyze the mean, minimum, maximum, and SD of the RD and RB between all simulated crimes and those synthetic crimes known to the police. We present analyses at the levels of OAs, LSOAs, MSOAs, and wards for four different crime types, in order to establish if the variability of the RD and RB becomes larger at more fine-grained spatial scales.

First, Table 6 presents the summary statistics of RD and RB for all crime types across the four spatial scales. On average, the RD is close to 62% at all the spatial scales (i.e., on average, 62% of crimes are unknown to police at each spatial scale), but the measures of dispersion—and the minimum and maximum values—vary considerably depending on the spatial level under study. The SD of the RD between all crimes and police-recorded offences is the largest at the level of OAs, whereas it is much smaller when crimes are aggregated at the LSOA level. It becomes almost zero at the level of MSOAs and wards. In other words, the RD has a large variability across small areas, but it is minimal when using larger geographies. In one OA, the police might be aware of the vast majority of crimes, and in another one, very few. Thus, geographic crime analysis produced solely from police records at highly localized spatial scales, such as OAs, and may show high concentrations of crime in some areas, but simply as an artefact of the variability in the crimes known to police. By contrast, the police know roughly the same proportion of crimes in all MSOAs and wards, with little variation around the mean. This is also observed in the minimum and maximum values. As such, documenting community differences in crimes based on police records aggregated at these larger scales will reduce the risk of mistakenly classifying some areas as high-crime density, but not others.

Similarly, the mean RB between all crimes and crimes known to police is roughly the same across all spatial scales, but the SD of the RB varies across levels of analysis. The SD is very large when crimes are aggregated at the level of OAs compared with larger scales.

Results shown in Table 6 , nevertheless, are produced from all crime types merged together and thus are likely to hide important heterogeneity depending on each type of crime under study. Crime research shows that different crime types are affected by different individual and contextual predictors (Andresen and Linning 2012 ; Quick et al. 2018 ), and there are also differences in terms of crime reporting to the police (Tarling and Morris 2010 ). Therefore, some crime types may be less affected by data biases than others, and it may be beneficial to disaggregate results by crime type in order to observe differences that may otherwise remain hidden.

Figure 1 shows boxplots of the RD between all crimes (known and unknown to police) and police-recorded crimes across crime types and spatial scales. Detailed results on this are also shown in Appendix Table 9 . We observe that, on average, the RD is lower for violent crimes than any other crime type. Thus, the proportion of total crime known to police is generally larger in the case of violent crimes. We also see that the measures of dispersion in the RD are much larger in the case of property crimes than all other crime types, while the variance of the RD of residence crimes appears to be the smallest. In the case of property crimes, for example, we observe that there is one OA with a RD equal to zero and another area with a RD equal to 100. In other words, in one OA, all property crimes were known to the police, while in the other small area not a single crime was known to police forces. Regardless of the crime type, larger levels of geography are associated with a smaller variance in the RD between areas, whereas the difference between the RD of crime aggregates for MSOAs or wards is generally small. In summary, geographic analysis produced from police records at larger spatial scales may show a more valid representation of the geographic distribution of crimes (known and unknown to police) than analysis produced for small areas.

Boxplots of RD% between all crimes and crimes known to police at the different spatial scales (simulated dataset)

In order to better illustrate the impact of selection bias on maps produced at the different spatial scales, Fig. 2 visualizes the values of RD between all property crimes and property crimes known to the police at the level of OAs, LSOAs, MSOAs, and wards in Manchester. We produce maps of property crimes since it is the crime type with the most extreme measures of dispersion in terms of RD, but similar—less extreme—results are also observed for other crime types. Figure 2 shows that the RD varies widely across OAs (i.e., in some areas, no crimes are known to police, and in others, nearly every crime is known to the police), while the RD between all crimes and police-recorded crimes becomes very homogeneous when crimes are aggregated at the scales of MSOAs and wards.

Maps of RD% between all property crimes and property crimes known to police at the different spatial scales (simulated dataset). Breaks based on equal intervals

Discussion and conclusions

Crime analysis and crime mapping researchers are moving toward increasingly fine-grained geographic resolutions to study the urban crime problem and to design spatially targeted policing strategies (Braga et al. 2018 ; Groff et al. 2010 ; Kirkpatrick 2017 ; Weisburd et al. 2012 ). Researchers document and explain community differences in crime to generate knowledge about crime patterns, test ideas, and assess interventions. Nevertheless, aggregating crimes known to police at such detailed levels of analysis increases the risk that the data biases inherent in police records reduce the accuracy of research outputs. These biases may contribute to the misallocation of police resources, and ultimately have an impact on the lives of those who reside in places mistakenly defined as high-crime-density or low-crime-density areas (Skogan 1977 ). They may also affect the validity of analyses which test theoretical explanations for the geographic distribution of crime (Gibson and Kim 2008 ).

This issue around the bias of police-recorded crime data largely depends on residents’ willingness to report crimes to police, and the police capacity to control places. Both are known to be affected by social and contextual conditions that are more prevalent in some areas than others (Berg et al. 2013 ; Goudriaan et al. 2006 ; Jackson et al. 2013 ; Slocum et al. 2010 ; Xie and Lauritsen 2012 ). The demographic and social characteristics of micro-places are usually very homogeneous (Brattbakk 2014 ; Oberwittler and Wikström 2009 ), which means that populations unwilling to report crime and cooperate with the police will concentrate in particular places, while other areas may contain social groups that are much more inclined to report crime and work with the police. The influence of these factors is reduced when crimes are aggregated to meso- and macro-levels of spatial analysis with more heterogeneous populations. Our simulation study shows that aggregates of police-recorded crime produced for neighborhoods and wards show a much more accurate—less biased—image of the geography of crime compared with those aggregated to small areas. This can be attributed to greater variability (i.e., between-unit heterogeneity) in the proportion of crimes known to police at fine-grained spatial scales. This study also demonstrates that some crime types are affected by data bias differently, which demonstrates the need to disaggregate analyses by crime types.

However, our simulation study is also affected by some limitations that could be addressed in future research. Namely, our simulated dataset of crimes captures area victimization rates instead of area crime rates and, as a consequence, the empirical evaluation when comparing synthetically generated crimes with actual crimes known to GMP showed that our synthetic dataset could be further improved in those areas with a large difference between workday and residential populations. In order to mitigate against this shortcoming, future research should investigate replicating this analysis using census data for workday populations instead of census data for residential populations. This may allow for the generation of more accurate crime counts, especially in non-residential places where crime is prevalent, such as the city center and commercial districts. Moreover, since the CSEW sample in Manchester is very small, our approach assumed that slopes observed in regressions estimated from the CSEW at a national level apply to crimes in Manchester. Future research may merge several editions of the CSEW to obtain a large enough sample in Manchester. Nevertheless, in such a case, survey and census data would refer to different time periods, and there would be a risk of repeated respondents in the CSEW. There are three further limitations that may have more difficult solutions: (a) the CSEW and most victimization surveys do not record information of so-called victimless crimes (e.g., drug-related offences, corporate crimes) and homicides, for which generating synthetic estimates may be more complicated; (b) the sample of the CSEW consists of adults aged 16 or more, and thus it may be difficult to accurately generate crimes faced by individuals younger than 16 years; and (c) the census is only conducted every 10 years and generating periodic synthetic populations to estimate crime will require the implementation of novel techniques (e.g., spatial microsimulation models; Morris and Clark 2017 ). Future research will also explore the use of other individual and contextual variables recorded in the census and other data sources to further improve the precision of synthetic crime data. Moreover, this approach could be applied to other urban areas with available local crime surveys (e.g., Islington Crime Survey, Metropolitan Police Public Attitudes Survey) which would allow for an empirical evaluation of synthetic crime data generated in each local area.

Those who advocate the need for documenting and explaining micro-level community differences in crime have well-sustained arguments to claim that aggregating crimes at fine-grained levels of spatial analysis allows for better explanations of crime, and more targeted operational policing practices. To mention only a few of their arguments, Oberwittler and Wikström ( 2009 ) show that between-neighborhood crime variance and the statistical power of research outputs increase when smaller units of analysis are used; Steenbeek and Weisburd ( 2016 ) show that most temporal variability in crimes known to police can be attributed to micro-scales; Braga et al. ( 2018 ) show that increasing police control in high-crime-density areas reduces the overall prevalence and incidence of crimes; and Weisburd et al. ( 2012 ) argue that the social systems relevant to understanding the crime problem concentrate in small units of geography. It is not our intention to dismiss the merits of micro-level geographic crime analysis, nor do we directly assess whether the claims made by the advocates of micro-level mapping remain verifiable when analyzing unbiased datasets of crime (this is, perhaps, an area for future research). That said, the results reported in this paper serve to raise awareness about an important shortcoming of micro-level crime analysis. There is a clear need for academics and police administrations to evaluate whether crime rates are associated with conditions external to victimization. In particular, there is a need to make this evaluation with consideration for the spatial scale being used (Ramos et al. 2020 ). The potential sources of bias in police-recorded crime data should always be investigated and acknowledged with this in mind. Further efforts might focus on developing techniques which mitigate against these sources of bias to ensure that geographic crime analysis remains an effective tool in understanding and tackling the crime problem.

Aebi, M. F., & Linde, A. (2012). Conviction statistics as an indicator of crime trends in Europe from 1990 to 2006. European Journal on Criminal Policy and Research, 18 , 103–144.

Article   Google Scholar  

Andresen, M. A., & Linning, S. J. (2012). The (in)appropriateness of aggregating across crime types. Applied Geography, 35 (1-2), 275–282.

Baumer, E. P. (2002). Neighborhood disadvantage and police notification by victims of violence. Criminology, 40 (3), 579–616.

Berg, M. T., Slocum, L. A., & Loeber, R. (2013). Illegal behaviour, neighborhood context, and police reporting by victims of violence. Journal of Research in Crime and Delinquency, 50 (1), 75–103.

Bivand, R., Müller, W. G., & Reder, M. (2009). Power calculations for global and local Moran’s I. Computational Statistics & Data Analysis, 53 (8), 2859–2872.

Bowers, K., & Johnson, S. D. (2014). Crime mapping as a tool for security and crime prevention. In M. Gill (Ed.), The Handbook of Security (2nd ed., pp. 566–587). London: Palgrave Macmillan.

Chapter   Google Scholar  

Braga, A. A., Weisburd, D., & Turchan, B. (2018). Focused deterrence strategies and crime control: An updated systematic review and meta-analysis of the empirical evidence. Criminology & Public Policy, 17 (1), 205–250.

Brantingham, P. J. (2018). The logic of data bias and its impact on place-based predictive policing. Ohio State Journal of Criminal Law, 15 (2), 473–486.

Google Scholar  

Brantingham, P. L., & Brantingham, P. J. (1995). Criminality of place. Crime generators and crime attractors. European Journal of Criminal Policy and Research, 3 , 5–26.

Brantingham, P. L., & Brantingham, P. J. (2004). Computer simulation as a tool for environmental criminologists. Security Journal, 17 , 21–30.

Brattbakk, I. (2014). Block, neighbourhood or district? The importance of geographical scale for area effects on educational attainment. Geografiska Annaler: Series B, Human Geography, 96 (2), 109–125.

Britt, C. L., Rocque, M., & Zimmerman, G. M. (2018). The analysis of bounded count data in criminology. Journal of Quantitative Criminology, 34 , 591–607.

Bruinsma, G. J. N., & Johnson, S. D. (Eds.). (2018). The Oxford handbook of environmental criminology . New York: Oxford University Press.

Buil-Gil, D., Medina, J., & Shlomo, N. (2021). Measuring the dark figure of crime in geographic areas. Small area estimation from the Crime Survey for England and Wales. British Journal of Criminology, 61 (2), 364–388.

Chaiken, J. M. & Rolph, J. E. (1981). Methods for estimating crime rates of individuals. The Rand Corporation. Report R-2730//1-NIJ . Retrieved from https://www.rand.org/content/dam/rand/pubs/reports/2009/R2730.1.pdf .

Compton, G., Wilson, A., & French, B. (2017). The 2011 census: From preparation to publication. In J. Stillwell (Ed.), The Routledge handbook of census resources, methods and applications (pp. 33–53). London: Routledge.

de Candolle, A. (1830 [1987a]). Considérations sur la statistique des délits. Déviance et Société , 11 (4), 352–355.

de Candolle, A. (1832 [1987b]). De la statistique criminelle. Déviance et Société , 11 (4), 356-363.

Gale, C., Singleton, A., & Longley, P. (2017). Creating a new open geodemographic classification of the UK using 2011 census data. In J. Stillwell (Ed.), The Routledge handbook of census resources, methods and applications (pp. 213–229). London: Routledge.

Gibson, J., & Kim, B. (2008). The effect of reporting errors on the cross-country relationship between inequality and crime. Journal of Development Economics, 87 (2), 247–254.

Goudriaan, H., Wittebrood, K., & Nieuwbeerta, P. (2006). Neighbourhood characteristics and reporting crime: Effects of social cohesion, confidence in police effectiveness and socio-economic disadvantage. British Journal of Criminology, 46 (4), 719–742.

Groff, E. R., & Mazerolle, L. (2008). Simulated experiments and their potential role in criminology and criminal justice. Journal of Experimental Criminology, 4 , 187.

Groff, E. R., Weisburd, D., & Yang, S. M. (2010). Is it important to examine crime trends at a local “micro” level?: A longitudinal analysis of street to street variability in crime trajectories. Journal of Quantitative Criminology, 26 , 7–32.

Hart, T. C. & Rennison, C. (2003). Reporting crime to the police, 1992-2000. Special Report. Bureau of Justice Statistics . Retrieved from https://static.prisonpolicy.org/scans/bjs/rcp00.pdf .

HMIC. (2014). Crime-recording: Making the victim count. The final report of an inspection of crime data integrity in police forces in England and Wales. HMIC Report . Retrieved from https://www.justiceinspectorates.gov.uk/hmicfrs/wp-content/uploads/crime-recording-making-the-victim-count.pdf

Jackson, J., Bradford, B., Stanko, B., & Hohl, K. (2013). Just authority? Trust in the police in England and Wales . Abingdon: Routledge.

Kirkpatrick, K. (2017). It’s not the algorithm, it’s the data. Communications of the ACM, 60 (2), 21–23.

Kitsuse, J. I., & Cicourel, A. V. (1963). A note on the uses of official statistics. Social Problems, 11 (2), 131–139.

Lee, Y., Eck, J. E., & O, S. & Martinez, N. N. (2017). How concentrated is crime at places? A systematic review from 1970 to 2015. Crime Science, 6 (6).

Manchester City Council. (2011). Workday population summary: 2011 census. Manchester City Council Report . Retrieved from https://www.manchester.gov.uk/download/downloads/id/25545/q05q_2011_census_summary_-_workday_population.pdf .

McCandless, R., Feist, A., Allan, J. & Morgan, N. (2016). Do initiatives involving substantial increases in stop and search reduce crime? Assessing the impact of Operation BLUNT 2. Home Office Report . Retrieved from https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/508661/stop-search-operation-blunt-2.pdf

McLennan, D., Barnes, H., Noble, M., Davies, J., Garratt, E. & Dibben, C. (2011). The English Indices of Deprivation 2010. Department for Communities and Local Government . Retrieved from: https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/6320/1870718.pdf

Moretti, A. (2020). Simulation studies. In I. P. Atkinson, S. Delamont, A. Cernat, R. Williams, & J. Sakshaug (Eds.), SAGE Research Methods Foundations . SAGE.

Morris, M. A., & Clark, S. (2017). A big data application of spatial microsimulation for neighborhoods in England and Wales. In L. Schintler & Z. Chen (Eds.), Big data for regional science . London: Routledge.

Oberwittler, D., & Wikström, P. O. H. (2009). Why small is better: Advancing the study of the role of behavioral contexts in crime causation. In D. Weisburd, W. Bernasco, & G. J. N. Bruinsma (Eds.), Putting crime in its place: Units of analysis in geographic criminology (pp. 35–60). New York: Springer.

Office for National Statistics. (2013). Crime Survey for England and Wales, 2011-2012 . UK Data Service, SN 7252. https://doi.org/10.5255/UKDA-SN-7252-2

Office for National Statistics. (2015). 2011 Census: General report for England and Wales. Office for National Statistics . Retrieved from https://www.ons.gov.uk/file?uri=/census/2011census/howourcensusworks/howdidwedoin2011/2011censusgeneralreport/2011censusgeneralreportforenglandandwalesfullreport_tcm77-384987.pdf .

Office for National Statistics. (2019). Recorded crime data at Community Safety Partnership and local authority level. Office for National Statistics . Retrieved from https://www.ons.gov.uk/peoplepopulationandcommunity/crimeandjustice/datasets/recordedcrimedataatcommunitysafetypartnershiplocalauthoritylevel .

Office for National Statistics (2020). Crime Severity Score (experimental statistics). Office for National Statistics . Retrieved from https://www.ons.gov.uk/peoplepopulationandcommunity/crimeandjustice/datasets/crimeseverityscoreexperimentalstatistics .

Pebesma, E. (2018). Simple features for R: standardized support for spatial vector data. The R Journal, 10 (1), 439–446.

Pierce, G. L., Spaar, S., & Briggs, L. R. (1988). The character of police work: Strategic and tactical implications . Boston: Northeastern University.

Quick, M., Li, G., & Brunton-Smith, I. (2018). Crime-general and crime-specific spatial patterns: A multivariate spatial analysis of four crime types at the small-area scale. Journal of Criminal Justice, 58 , 22–32.

R Core Team. (2020). R: A language and environment for statistical computing. R Foundation for Statistical Computing . Retrieved from https://www.R-project.org/ .

Ramos, R. G., Silva, B. F. A., Clarke, K. C., & Prates, M. (2020). Too fine to be good? Issues of granularity, uniformity and error in spatial crime analysis. Journal of Quantitative Criminology .

Schnebly, S. (2008). The influence of community-oriented policing on crime-reporting behaviour. Justice Quarterly, 25 (2), 223–251.

Sellin, T. (1931). The basis of a crime index. Journal of Criminal Law and Criminology, 22 (3), 335–356.

Sherman, L. W., Gartin, P. R., & Buerger, M. E. (1989). Hot spots of predatory crime: Routine activities and the criminology of place. Criminology, 27 (1), 27–55.

Skogan, W. S. (1974). The validity of official crime statistics: An empirical investigation. Social Science Quarterly, 55 , 25–38.

Skogan, W. G. (1977). Dimensions of the dark figure of unreported crime. Crime & Delinquency, 23 (1), 41–50.

Slocum, L. A., Taylor, T. J., Brick, B. T., & Esbensen, F. A. (2010). Neighborhood structural characteristics, individual-level attitudes, and youths’ crime reporting intentions. Criminology, 48 (4), 1063–1100.

Steenbeek, W., & Weisburd, D. (2016). Where the action is in crime? An examination of variability of crime across different spatial units in The Hague, 2001–2009. Journal of Quantitative Criminology, 32 , 449–469.

Tarling, R., & Morris, K. (2010). Reporting crime to the police. British Journal of Criminology, 50 (3), 474–490.

Tiefelsdorf, M. (2000). Modelling spatial processes. The identification and analysis of spatial relationships in regression residuals by means of Moran’s I . Berlin: Springer.

Tompson, L., Johnson, S., Ashby, M., Perkins, C., & Edwards, P. (2015). UK open source crime data: accuracy and possibilities for research. Cartography and Geographic Information Science, 42 (2), 97–111.

Townsley, M., & Birks, D. J. (2008). Building better crime simulations: systematic replication and the introduction of incremental complexity. Journal of Experimental Criminology, 4 , 309–333.

UK Statistics Authority. (2014). Assessment of compliance with the code of practice for official statistics. Statistics on crime in England and Wales. Assessment Report 268 . Retrieved from https://www.statisticsauthority.gov.uk/wp-content/uploads/2015/12/images-assessmentreport268statisticsoncrimeinenglandandwale_tcm97-43508-1.pdf .

Weisburd, D. (2015). The law of crime concentration and the criminology of place. Criminology, 53 (2), 133–157.

Weisburd, D., Bruinsma, G. J. N., & Bernasco, W. (2009). Units of analysis in geographic criminology: Historical development, critical issues, and open questions. In D. Weisburd, W. Bernasco, & G. J. N. Bruinsma (Eds.), Putting crime in its place: Units of analysis in geographic criminology (pp. 3–34). New York: Springer.

Weisburd, D., Groff, E. R., & Yang, S. M. (2012). The criminology of place. Street segments and our understanding of the crime problem . New York: Oxford University Press.

Book   Google Scholar  

Weisburd, D., & Lum, C. (2005). The diffusion of computerized crime mapping in policing: Linking research and practice. Police Practice and Research, 6 (5), 419–434.

Xie, M. (2014). Area differences and time trends in crime reporting: Comparing New York with other metropolitan areas. Justice Quarterly, 31 (1), 43–73.

Xie, M., & Baumer, E. P. (2019a). Neighborhood immigrant concentration and violent crime reporting to the police: A multilevel analysis of data from the National Crime Victimization Survey. Criminology, 57 (2), 237–267.

Xie, M., & Baumer, E. R. (2019b). Crime victims’ decisions to call the police: Past research and new directions. Annual Review of Criminology, 2 , 217–240.

Xie, M., & Lauritsen, J. (2012). Racial context and crime reporting: A test of Black’s stratification hypothesis. Journal of Quantitative Criminology, 28 , 265–293.

Download references

Acknowledgements

The authors would like to thank Reka Solymosi for comments that greatly improved the manuscript.

This work is supported by the Campion Grant of the Manchester Statistical Society (project title: “Mapping the bias of police records”).

Author information

Authors and affiliations.

School of Social Sciences, University of Manchester, Humanities Bridgeford Street building, 176 Oxford Road, Manchester, M13 9PL, UK

David Buil-Gil

Manchester Metropolitan University, Manchester, UK

Angelo Moretti

University of Leeds, Leeds, UK

Samuel H. Langton

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to David Buil-Gil .

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

To estimate whether each crime took place in the victims’ local area or somewhere else, we follow the same procedure as in step 3. First, we estimate a logistic regression model of crimes happening in the local area (as opposed to crimes happening elsewhere) from the CSEW dataset of crimes. We use the same individual independent variables as above (see model results in Table 8 ). Second, we estimate whether each simulated crime took place in the resident’s local area or somewhere else following a Bernoulli distribution from the regression coefficient estimates presented in Table 8 and the independent variables simulated in Step 1. For example, to estimate whether vehicle crime j suffered by person i took place in local area, denoted by AVehi ji , we compute:

where \( {\hat{\beta}}_p \) is the regression model coefficient estimate for a p independent variable.

Then, we remove all those offences that did not take place in the local area from our synthetic dataset of crimes.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Buil-Gil, D., Moretti, A. & Langton, S.H. The accuracy of crime statistics: assessing the impact of police data bias on geographic crime analysis. J Exp Criminol 18 , 515–541 (2022). https://doi.org/10.1007/s11292-021-09457-y

Download citation

Accepted : 12 January 2021

Published : 26 March 2021

Issue Date : September 2022

DOI : https://doi.org/10.1007/s11292-021-09457-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Crime analysis
• Official Statistics
• Unreliability
• Simulation experiment
• Find a journal
• Publish with us
• Track your research

To read this content please select one of the options below:

Please note you do not have access to teaching notes, using crime data in academic research: issues of comparability and integrity.

Records Management Journal

ISSN : 0956-5698

Article publication date: 16 November 2015

Research into crime is reliant on data that is recorded and published by criminal justice agencies; data which is collected for other purposes. Considering the suitability of geocoded crime data for academic research purposes, this paper will demonstrate the difficulties faced regarding the availability, integrity and reliability of readily accessible criminal justice data.

Design/methodology/approach

Data from two countries – England and Germany – were considered and set in a wider European Union (EU) context. Using the data received from requests made to the Freedom of Information Act (FOIA) in England and openly published reports and data available from Germany, the authors provide a contextual picture of the availability and operability of data recorded by these agencies. Geocoded data that enable cross-national comparisons with respect to immigration, ethnicity and crime are particularly hard to locate, and conducting research using data (such as crime data) whose “integrity” is questionable in an academic environment becomes increasingly problematic.

Analysing secondary data produced by a number of agencies are amplified due to the different methods of collection, management, retention and dissemination. It was found that even within England, the information provided by police forces varied greatly. Data in Germany were found to be more openly available and published electronically by a number of different criminal justice agencies; however, many of the issues apparent in English data regarding data integrity were also identified here.

Originality/value

The need for good record-keeping and information sharing practices has taken on added significance in today’s global environment. The better availability of comparable criminal justice data has the potential to provide academics with increased opportunities to develop an evidence base for policymaking.

• Data management
• Freedom of information
• Comparability
• Data availability
• Data integrity

Acknowledgements

The authors would like to thank the Prevention of and Fight against Crime Programme of the European Commission Directorate-General Home Affairs for funding this research. The views expressed in this publication are solely those of the authors and do not indicate agreement or acceptance by the research fund provider. The European Commission is not responsible for any use that may be made of the information contained therein. The authors would also like to thank the reviewers for their helpful comments and suggestions to improve the article.

Ludwig, A. and Marshall, M. (2015), "Using crime data in academic research: issues of comparability and integrity", Records Management Journal , Vol. 25 No. 3, pp. 228-247. https://doi.org/10.1108/RMJ-05-2015-0017

Emerald Group Publishing Limited

Copyright © 2015, Emerald Group Publishing Limited

Related articles

We’re listening — tell us what you think, something didn’t work….

Report bugs here

All feedback is valuable

Please share your general feedback

Join us on our journey

Platform update page.

Visit emeraldpublishing.com/platformupdate to discover the latest news and updates

Questions & More Information

Answers to the most commonly asked questions here

4 Chapter 4: Survey Research

Case study: national crime victimization survey.

Research Study

Criminal Victimization in the United States 1

Research Question

How many violent and property crime victimizations occurred in the United States in 2010?

Methodology

The results of the National Crime Victimization Survey (NCVS) are provided on an annual basis by the U.S. Census Bureau and serve as an alternative to the Uniform Crime Reports. The NCVS collects information on the violent crimes of rape or sexual assault, robbery, aggravated assault, and simple assault, and the property crimes of household burglary, motor vehicle theft, and theft. The results are derived from a nationally representative sample of U.S. households selected through a stratified, multistage cluster sampling process (see Chapter 3). The survey results are based on data gathered from residents living throughout the United States, including persons living in group quarters, such as dormitories, rooming houses, and religious group dwellings. Armed Forces personnel living in military barracks and institutionalized persons, such as prison inmates, are not included in the survey.

Each housing unit selected for the NCVS remains in the sample for 3 years, with each of the seven interviews taking place at 6-month intervals. An NCVS interviewer’s first contact with a household selected for the survey is in person and the survey is completed through a face-to-face interview. The interviewer may conduct subsequent interviews by telephone. To elicit more accurate reporting of incidents, the NCVS uses a self-report method that includes a direct interview with each person 12 years or older in the household. In 2010, 40,974 households and 73,283 individuals age 12 and older were interviewed for the NCVS. Each household was interviewed twice during the year. The response rate was 92.3% of households and 87.5% of eligible individuals.

The NCVS has been collecting data on personal and household victimization since 1973. The data include type of crime, month, time and location of the crime, relationship between victim and offender, characteristics of the offender, self-protective actions taken by the victim during the incident and results of those actions, consequences of the victimization, type of property lost, whether the crime was reported to the police and reasons for reporting or not reporting, and offender use of weapons, drugs, and alcohol. Basic demographic information such as age, race, gender, and income is also collected to enable analysis of victimization by various subpopulations.

During 2010, U.S. residents age 12 or older experienced an estimated 18.7 million violent and property crime victimizations, down from 20.1 million in 2009 and 24.2 million in 2001. The criminal victimizations in 2010 included an estimated 3.8 million violent victimizations and 14.8 million property victimizations. Violent and serious violent victimizations (includes rape or sexual assault, robbery, and aggravated assault) declined by nearly 34% between 2001 and 2010.

From 2001 to 2010, weapon violence (26% to 22%) and stranger-perpetrated violence (44% to 39%) declined. Between 2001 and 2010, about 6% to 9% of all violent victimizations were committed with firearms. This percentage has remained stable since 2004. The percentage of victims of violent crimes who suffered an injury during the victimization increased from 24% in 2008 to 29% in 2010. About 50% of all violent victimizations and nearly 40% of property crimes were reported to the police in 2010. These percentages have remained stable over the past 10 years. Males (15.7 per 1,000 population) and females (14.2 per 1,000 population) had similar rates of violent victimization during 2010.

Limitations with the Study Procedure

While rape or sexual assault significantly increased from 2009 to 2010, you should be cautious in interpreting this change because the estimates of rape/sexual assault are based on a small number of cases reported to the interviewers. Small changes in the number of victimizations can result in large year-to-year percentage changes. For instance, the 2010 estimate of rape or sexual assault is based on 57 reported cases compared to 36 reported cases in 2009. This represents an increase of 21 actual cases but also represents a 49.6% increase in the national estimate of rape or sexual assault from 2009 to 2010. The measurement of rape or sexual assault represents one of the most serious challenges in the field of victimization research. Rape and sexual assault remain sensitive subjects that are difficult to ask about in the survey context.

Any time national estimates are derived from a sample rather than the entire population, as is the case with the NCVS, because of sampling error it is important to be cautious when drawing conclusions about the size of one population estimate in comparison to another (e.g., does aggravated assault occur more frequently than robbery?) or about whether population estimates are changing over time (e.g., did robbery increase or decrease in the past year?). Although one figure may be larger than another, the NCVS estimates are based on responses from a sample of the population. Therefore, each estimate has some degree of sampling error. The sampling error of a population estimate depends on several factors, including the amount of variation in the responses, the size and representativeness of the sample, and the size of the subgroup for which the estimate is computed, as illustrated in the rape or sexual assault example previously discussed.

In addition to sampling error, the NCVS results are subject to non-sampling error. While substantial care is taken in the NCVS to reduce the sources of non-sampling error throughout all the survey operations, an unknown amount of non-sampling error still remains. A major source of non-sampling error includes the inability of the respondents to recall in detail the crimes that occurred during the six months prior to the interview. The NCVS uses a 6-month reference period. Respondents are asked to report victimization experiences occurring in the last 6 months. Generally, respondents are able to recall more accurately an event that occurred within 3 months of the interview rather than one that occurred within 6 months. Research indicates that assault is recalled with the least accuracy of any crime measured by the NCVS. This may be related to the tendency of victims to not report crimes committed by offenders who are not strangers, especially if they are relatives. Recall problems may result in an understatement of the actual rate of assault.

However, a shorter reference period would require more field interviews per year, increasing the data collection costs significantly. These increased costs would have to be balanced by cost reductions elsewhere (sample size is often considered). Reducing sample size however, reduces the precision of estimates of relatively rare crimes such as rape or sexual assault. In light of these trade-offs of cost and precision, a reference period of 6 months is used for the NCVS.

Other sources of non-sampling error can result from other types of response mistakes, including errors in reporting incidents as crimes, misclassification of crimes, systematic data errors introduced by the interviewer, and errors made in coding and processing the data.

Impact on Criminal Justice

The NCVS is one of the two national sources of crime data in the United States. The NCVS is generally viewed as a more reliable and valid measure of crime than the Uniform Crime Reports. The data from the NCVS survey are particularly useful for calculating crime rates, both aggregated and disaggregated, and for determining changes in crime rates from year to year. In addition, the NCVS is the primary source of information on the characteristics of criminal victimization and on the number and types of crimes not reported to law enforcement authorities. It provides the largest national forum for victims to describe the impact of crime and the characteristics of violent offenders.

In This Chapter You Will Learn

Why nonresponse is a major source of survey error and how nonresponse impacts survey results

The four main mechanisms for assessing validity as well as how to increase the validity of survey questions and responses

How to assess the reliability of survey questions and responses as well as ways to increase reliability

The methods in which surveys are distributed and the strengths and weaknesses of each

Introduction

Survey research involves the collection of information from individuals through their responses to questions, and represents one of the most widely used research tools in crime and criminal justice studies. Surveys are one of the most common ways to obtain information about attitudes and behaviors and involve collecting information from people via self-administered surveys, telephone interviews, face-to-face interviews, and more recently Internet and e-mail–based surveys.

In survey research, the main way of collecting information is by asking people questions. Their answers constitute the data to be analyzed. For example, the U.S. Census is a self-administered survey where individuals receive surveys in the mail and are asked to fill them out and send them back to the U.S. Census Bureau. In addition to the counting of people, the U.S. Census collects data about the race, age, household composition, education, type of housing, and many other characteristics of the people counted. As presented in the introductory case study, the NCVS is another example of a large-scale survey where data is collected via face-to-face and telephone interviews.

Most people are familiar with surveys, especially those that involve the measurement of public opinion for newspaper and magazine articles, the measurement of political perceptions and opinions involving politicians, and marketing research designed to understand consumer preferences and interests. In fact, we are inundated with surveys in our daily lives. From the restaurant that offers you a free appetizer on a future visit for completing a telephone survey about your experience, to the call center that wants your feedback on the performance of the customer service representative at the conclusion of your phone call, to solicitations to complete online surveys regarding your website experience, to the evaluation of your professor at the end of the semester, we don’t have to look far for an example of a survey in our daily lives.

The same is true for research in criminology and criminal justice. Surveys are frequently the data collection method of choice to answer research questions. For example, if we want to know whether the police officers in our local police department prefer to work 8-, 10-, or 12-hour shifts, we can survey them to address this issue. If we want to determine the level of fear of crime on a college campus, we can survey students, faculty, staff, and visitors to gauge their level of fear of crime. If we want to know if the public agrees with recent U.S. Supreme Court decisions, we can survey residents and ask them their opinion. If we want to understand how correctional officers view their roles and interactions with inmates, we can survey the officers regarding the topic. If we want to identify the extent, causes, and consequences of bullying, we can survey students to better understand bullying. Overall, surveys can be used to answer a wide range of research questions involving crime and the operation of the criminal justice system.

Three characteristics of surveys make them a preferable source of data collection in comparison to other sources. 2 First, probability sampling is typically used in surveys, which enables the research consumer to have confidence that the sample is not a biased one and is a reflection of the larger population. Data from research studies using a probability sample are definitely preferred over data from a non-probability sample, such as those who attend a college class or happen to be convenient to survey. Second, standardized measurement (i.e., each respondent is asked the same questions in the same manner) ensures that comparable information is obtained about everyone who responds to the survey. Standardized measurement produces meaningful statistics and comparisons of respondents (e.g., how are juveniles who are chronically truant different from juveniles who are not?). Third, the development of a survey ensures that all the data needed for a given analysis are available. For example, you may find in your literature that parental supervision is an important factor that distinguishes truants from nontruants. Since you are creating the survey, you can be sure to ask questions about the respondent’s level of parental supervision so the relationship between truancy and parental supervision can be addressed in your study.

Survey Components

Surveys bring together sampling, question design, and methods of survey distribution. It is important to note that each of these activities has many applications outside of surveys, but their presence is essential to quality surveys. For consumers of research, it is important to understand how the details of each component of a survey can affect its precision, consistency, and accuracy (i.e., its reliability and validity). How the sample is selected, which questions are asked, and the procedures used to collect the answers all impact the quality of the survey and its results. 3

Sampling With the U.S. Census as an exception, survey information is typically collected from a sample of the population, rather than from every member of the population. The ability to select a sample that is representative of the whole population was a significant development that made surveys a useful research tool. The keys to good sampling are finding a way to give all (or nearly all) members of the population the same (or a known) chance of being selected and using probability sampling methods for choosing the sample. 4

Every survey involves a number of decisions that have the potential to enhance or detract from the accuracy of the survey results. With respect to sampling, critical issues include the following: the choice of whether or not to use a probability sample, the size of the sample, the sampling technique utilized, and the rate of response (the percentage of those sampled who respond to the survey). Since sampling was covered in detail in Chapter 3, it will not be revisited in this chapter. At this point, realize that probability sampling techniques are typically used in surveys. Therefore, the material covered in Chapter 3 is applicable to survey development and distribution. This chapter will discuss the response rate for surveys since it directly applies to surveys and was not covered in Chapter 3.

Survey Question Design In surveys, the questions measure the concepts and variables under study. Survey question design is a critical element of survey research. Sticking with the theme of this book regarding the development of educated consumers of research, the focus on question design in this chapter will not be on how to write survey questions but on strategies for evaluating the quality of the questions. Survey researchers evaluate questions to find out if they are well understood by the respondents and if the answers are meaningful. In other words, the reliability and validity of the survey questions must be assessed in determining the quality of the survey. The researcher must decide the extent to which previous literature regarding the reliability and validity of questions will be drawn upon in the development of the survey as well as the process for question evaluation.

Methods of Survey Distribution Surveys can be distributed through several means, including face-to-face or telephone interviews or through self-administered means via the mail or Internet. Some surveys have respondents answer self-administered questions while others use an interviewer to ask questions and record answers. When interviewers are used, it is important to avoid having them influence the answers given by the respondents. The decision about which mode of data collection to use has important cost implications and affects the quality of the data that will be collected. 5

The three components of survey development (i.e., sampling, question design, and methods of distribution) are interrelated because the quality of the survey data will not be better than the most error-prone feature of the survey design. A large sample size will not make up for a low rate of response. Similarly, a high response rate will not compensate for unreliable and invalid survey questions. Survey researchers must focus on each component of the survey, including sampling, question design, and method of distribution, because weaknesses in one area cannot be rectified by the strengths of another area.

Since sampling has already been covered in Chapter 3, this chapter discusses question design and methods of survey distribution. A concentrated effort is made to discuss the realities and the practical problems with the decisions made by survey researchers. The goal is to provide a comprehensive overview of the sources of error in surveys and the range of methodological issues that arise when using surveys. Although survey research is a common way to solicit information from a wide variety of individuals, it is not without its limitations, which will be discussed throughout this chapter. We begin our discussion with an overview of the sources of error in survey research.

Issues in Survey Development: Nonresponse and Survey Design

Sampling error, as discussed in Chapter 3, occurs because the sample drawn from a population will not exactly match the population. For example, survey results from a sample of citizens on attitudes toward the use of the death penalty will not be identical to the overall survey results if an entire state population of citizens was surveyed. As discussed in Chapter 3, the difference in survey results between a sample and a population is called sampling error. Researchers expect there to be a difference between the sample results and the results from an entire population, even when the sample is representative of the population. They just try to minimize the error as much as possible.

When a sample is drawn, it is inevitable that the sample will differ from the population from which it was drawn. The way the sample is drawn (e.g., probability vs. non-probability sampling) can affect how closely the sample is likely to mirror the characteristics of the population. Regardless of the sophistication of the sampling strategy, by chance, the sample will differ slightly from what it would look like if it was an exact mirror image of the population. One of the goals of survey research is to minimize the random, by chance differences between the sample and the population.

Nonresponse

Nonresponse is a major source of survey error. There are three categories of nonrespondents (i.e., those selected to be in a sample who do not actually respond to the survey) 6 :

1. Those who did not receive the survey, thereby not giving them a chance to respond to the survey. For example, this includes mailed surveys that are undeliverable due to bad addresses and telephone surveys that fail to call when the potential respondent is available to answer the phone.

2. Those who receive the survey but refuse to complete it. For example, this includes potential respondents who refuse to participate in the NCVS even though their household has been selected for participation. In 2010, 7.7% of the sampled households refused to participate in the survey. 7

3. Those asked to participate in the survey but are unable to perform the task required of them. This includes people who are too ill to be interviewed, who have language barriers, or whose reading and writing skills preclude them from completing a self-administered survey.

Failure to collect survey data from a high percentage of those selected to be in a sample is a major source of survey error. 8 When it comes to surveys, the quality of the results not only depends on how well the sample is a close approximation of the population but also how well those who respond to the survey reflect the total population. The method of distribution that a researcher decides to use can have a major influence on the response rate and the extent to which nonrespondents introduce bias into the results. As discussed later in this chapter, some methods of distribution (e.g., face-to-face interviews) have higher response rates than other distribution methods (e.g., mailed surveys). Overall, a low response rate will bias the sample, which means that the nonresponse makes the respondents systematically different from the population from which they were drawn, thereby producing biased results. For example, let’s say we surveyed 100 physicians on the effects of energy drinks on general health, and 50 returned the survey, for a response rate of 50%. The question we should consider is, “How do the 50% who did not respond differ from the 50% who did?” In this circumstance, if we had the other 50% of the responses, they might totally change the results of the study.

Calculating Response Rates In assessing the quality of a research project that used a survey to collect data, the response rate is a basic parameter for evaluating a project. Simply, the response rate is the number of people who respond to the survey divided by the number of people sampled. The denominator includes all people in the sample who were selected as a potential survey respondent but did not respond for whatever reason. This includes the three categories of nonrespondents previously listed, including those who do not receive the survey, those who receive the survey but refuse to participate, and those who are asked to participate in the survey but are unable to perform the task required of them.

Since the response rate is a simple way to assess the quality of a study involving a survey, researchers want to demonstrate a high response rate. To this end, some researchers do not include some of the categories of nonrespondents in the calculation of the response rate, which artificially inflates the response rate for the study. Most commonly, the category of nonrespondents that are excluded from the response rate calculation are those who did not receive the survey.

For example, let’s say that a sample of 1,000 college students has been selected for a survey, designed to be completed via telephone interviews, on criminal victimization on campus. In this fictitious study, 500 students completed the survey, 100 students refused to complete the survey and hung up on the interviewer, while another 400 never answered their phone despite numerous calls. What is the response rate? Based on the discussion above, the response rate is 50%. The response rate is the number of people who complete the survey (i.e., 500) divided by the number of people sampled (i.e., 1,000). However, some researchers will exclude the 400 potential respondents who were in the sample but never answered their phone despite repeated attempts. With this exclusion, the response rate is now 83.3%, which is much higher, and therefore better, because the researcher has only included those who completed the survey and those who refused to do so in the response rate (i.e., 500/600).

As an educated consumer of research, you need to be cautious when interpreting response rates. It is important to specifically know how the reported response rate was calculated. As illustrated in the example, differences in the way response rates are calculated can make comparisons across studies difficult. Instead of reporting a response rate, some researchers will report a “completion rate,” which excludes those who did not receive the survey from the calculation. In our above example, the completion rate is 83.3% because the 400 students who never answered the phone are excluded in the calculation. The completion rate will always be higher than the response rate as outlined above, which includes selected but uncontacted individuals in the denominator. Although a response rate seems simple to calculate, be sure to remember that it is important to understand how the response rate was calculated before you decide if the response rate is high.

The Impact of Nonresponse You may now be wondering what is an acceptable response rate. Babbie 9 has suggested that a 50% response rate is adequate, 60% is good, and 70% is very good. Besides focusing on the numeric value of the response rate and how it fits with Babbie’s categorization, it is more important to determine if those who did respond are a representative cross section of the sampled group than it is to have a high response rate. In other words, it is critical for the survey researcher to assess if there are significant differences between the respondents and the sample.

In addition, ideally, a researcher will have meaningful information on those who did not respond. Demographic variables such as education, race, ethnicity, gender, age, and income may be helpful in assessing differences between respondents and nonrespondents. An assessment could show that those who did respond to the survey were quite similar in terms of education, race, ethnicity, gender, age, and income to those who did not respond. If the respondents and nonrespondents are quite similar, a researcher could legitimately analyze the results of a survey even if the response rate was extremely low (e.g., 10%). Basically, the effect of nonresponse on survey results depends on the percentage not responding and the extent to which those not responding are biased (i.e., systematically different from the population). Again, the issue isn’t specifically about the response rate, but more critically about whether the respondents and nonrespondents are similar or different on key characteristics that are related to the research questions.

Another issue that needs to be considered in determining the adequacy of the response rate is the population studied. You would expect a higher response rate from a survey of students in the Introduction to Criminal Justice course at your university than from a citizen survey in the same community, even though the topic may be the same. The students probably have a preexisting relationship with the researcher, which, coupled with the use of a group-administered survey (discussed later in this chapter), will lead to a higher response rate in comparison to a survey that is mailed to community residents. In addition, individuals that have an interest in the survey topic are more likely to respond to a survey than those who are less interested. Therefore, you can expect a higher response rate from a survey of police chiefs in California regarding the role of municipal police departments in fighting terrorism than from a survey of California residents on the same topic. Therefore, the adequacy of a response rate is more complex than merely looking at the number and determining how it measures up to Babbie’s stated standards.

An example of significant bias from low response and poor sampling where the respondents were significantly different from the population is the classic Literary Digest presidential poll in 1936. The survey, which was administered via mail, predicted a victory for Alf Landon in an election that Franklin Roosevelt won by a huge margin. The sample of addresses for the mailed survey was selected from telephone books. At the time, not everyone had a telephone so most voters were not listed in the telephone book. In addition, Republicans (those in Landon’s party) were much more likely to have telephones in 1936. Furthermore, only a minority of those asked to return questionnaires did so, which also led to the inaccurate survey results.

Overall, nonresponse is a problematic source of survey error. As the response rate decreases, it becomes more likely that the nonrespondents will differ from the survey respondents. When there are significant differences between respondents and nonrespondents, each group represents biased subgroups of the total population. Bias means that in some systematic way the individuals responding to a survey are different from the overall population. Therefore, the results based on the survey respondents are not reflective of the overall population.

Although we can calculate a response rate, we usually do not know for sure the impact of nonresponse on the survey results. Perhaps the strongest argument for efforts to increase response rates is credibility. 14 When response rates are high, there is only a small potential for error due to nonresponse to have a significant impact on the survey results. When response rates are low, there is great potential for error due to nonresponse to have a significant impact on the survey results. Finally, a low response rate also provides critics of the survey results an easy, intuitive basis on which to say the data are not credible.

WHAT RESEARCH SHOWS: IMPACTING CRIMINAL JUSTICE OPERATIONS

What Causes Inmates to Riot? 10

Early research into the causes of prison riots focused primarily upon the deprivations that are part of the prison environment. Gresham Sykes 11 provided one of the first analyses of prison riots when he explored the causes of two riots that took place at the New Jersey State Prison in the spring of 1952. Sykes attributed the riots to changes in the inmate social system that had stripped the inmates of their control. Based on his observations, Sykes believed that the transfer of power from the inmate social system to prison staff exaggerated the deprivations experienced by inmates and contributed to the riots. His insights, however, failed to provide a complete understanding of why inmates rioted. Overcrowding, poor living conditions, lack of programming, guard brutality, poor administration, and a concentration of violence-prone inmates have all been cited as contributing factors to riots despite the fact that these conditions also exist in prisons where no riots have ever taken place. Further, Sykes’ explanations of prison riots were based on a case study (see Chapter 6) from a single prison facility where a riots had occurred.

In an effort to formulate a more comprehensive explanation for why inmates riot, Richard Wilsnack examined data from 48 state prisons. Included in his sample were prisons that had experienced a riot, some other type of collective inmate violence (e.g., work stoppage and hunger strikes), as well as prisons with no reported collective violence. He was the first to conduct a large-scale quantitative study of collective inmate violence. He created a 160-question survey that was mailed to 51 correctional institutions, including the largest state prison institution for each state and the District of Columbia. All of the prisons housed adult males convicted of felony crimes and were medium or maximum security facilities. Institutions were selected to be representative of the types of prisons where riots were most likely to occur. Only three states—Alabama, Kansas, and Mississippi—failed to respond to the survey, for a response rate of 94%. To improve the reliability of responses, questions were constructed in such a way that respondents had to provide only simple replies.

Twelve of the prisons surveyed reported a riot during the study period. The study identified several contributing factors to the riots. First, an analysis of prison conditions revealed that inmate deprivation, disorganization, and conflict existed in prisons where riots had taken place. Three-fourths of the prisons reporting a riot had experienced an increase in inmate assault and/or had at least one occurrence of an inmate assault on a guard. Second, administrative instability and conflict were related to prison riots. Three-fourths of the prisons reporting a riot had experienced major administrative changes (i.e., turnover or extended absences) and conflict among staff members. While a shortage of correctional staff was not related to the occurrence of a riot, low salaries and high staff turnover were found to be associated. Third, external pressures existed in prisons where riots had taken place. Wilsnack found that pressure and publicity from outside of the prison were also related to the occurrence of a riot. All of the riots had taken place in institutions where legislators and concerned citizens had tried to influence prison operations. In addition, 80% of these institutions had received some type of media coverage prior to the riot. Overall, riots were more likely to occur in maximum security prisons with overcrowding and unoccupied inmates. Furthermore, riots were more likely to occur in facilities where inmates of different ages, convictions, and prior records were all housed together. Wilsnack’s findings did not support Sykes’ contention that riots occurred as a result of a disruption of the inmate social structure.

Wilsnack’s quantitative study of inmate collective violence enhanced our understanding of prison riots and other forms of collective prison disturbances. Despite the difficulties in researching prison riots, our understanding of the causes has grown significantly over the past several decades. Researchers have uncovered many of the conditions associated with prison riots and several theories have been developed to explain their occurrence. Prison administrators today are better equipped to respond to prison riots with emergency response teams. These teams consist of officers trained in hostage negotiation and disturbance control.

This body of research has also helped prison administrators develop preventative measures to reduce the likelihood of riots. Regular security audits, consistent enforcement of rules and procedures, maintaining effective communication between inmates, staff, and administrators, providing inmates with structured activities and appropriate programs, and using inmate classification systems are all important strategies for preventing the occurrence of riots. 12 The number of prison riots has significantly declined since the 1970s, despite the large increases in the number of inmates. According to Useem and Piehl, 13 this trend can be attributed to more effective prison management.

Survey Question Design

Designing a good survey instrument involves selecting survey questions that are needed to answer your research questions. As discussed in Chapter 1, one of the initial steps in any research project is to develop a research question or questions. The same is true for survey research. Since responses to the survey questions are the data collected in surveys, the survey responses are used to answer your research questions. Therefore, it is critical that questions included in the survey are able to answer your research question. For example, if your research question is, “Is there a relationship between parental supervision and chronic truancy?” you need to ask survey questions that can measure the variables identified in the research question: parental supervision and chronic truancy. Designing a question for a survey instrument is designing a measure.

Besides the recognition that the survey questions must be able to be used to answer the research questions, how do researchers know what to ask on a survey? When building a questionnaire, a survey researcher has two sources for survey questions. First, the researcher can include survey questions that have been used in other surveys. When another researcher has already designed a quality set of questions to measure a variable in your research question, it is recommended that you use the items from the preexisting survey, with a few caveats. First, the existing survey questions must be reliable and valid, which will be discussed in the next section. For now, this basically means that the survey questions must be a quality measurement of the variable. For example, if you are creating a survey to address criminal victimization on your campus, a great starting point for developing the survey questions is to use the NCVS, covered in the chapter opening case study, as a template for your survey questions. You can certainly modify the questions to meet your specific needs, but the NCVS is a high-quality measurement of criminal victimization and so it is a reasonable place to start when creating the survey questions. Second, you need to provide appropriate citation to the original developer of the survey questions in the research manuscript. For example, Rosenberg 15 developed a measure of self-esteem that is the most often used measure of self-esteem in survey research. In order to use the 10-question measure of self-esteem or a modified version, the researcher must provide appropriate citation to its original source.

A good source for prior surveys on a wide range of topics is the Inter-University Consortium for Political and Social Research (ICPSR) at the University of Michigan. 16 Your university probably belongs to the ICPSR. ICPSR is a website depository where major social and political survey files are archived. If your university belongs to ICPSR, you can get codebooks for any study on file at ICPSR. The codebooks include the actual survey questions asked and the response formats used. The studies archived at the ICPSR cover a wide range of criminal justice and criminology topics, so it is a good starting place if you are ever tasked with developing a survey.

Second, when building a questionnaire, a researcher can include items that she has personally constructed. This is the favored option when a satisfactory measure of the variable does not already exist or the variables in the research question are unique. Despite the discussion above, do not think that every good question about a topic has already been asked. And do not think that the questions perfectly suiting your purposes have already been formulated. Finally, it is certainly reasonable to use both options (i.e., prior surveys and personally constructed items) when constructing a survey. As you are reading and evaluating survey research, the researcher should inform you about the source of the questions: whether based upon prior surveys or if questions were developed by the researcher.

RESEARCH IN THE NEWS

Want a Job? Check the Spelling on Your Facebook Profile. 1 7

When it comes to looking for jobs, you better put your best Facebook forward. Recruiters are looking, and frankly, they are not impressed with your poor grammar or posts about your latest unforgettable exploits, according to a new survey. The survey was conducted by social recruiting platform Jobvite. The company surveyed more than 1,000 human resources and recruitment professionals on their social recruiting activities and intentions with over 800 responding to the survey. Respondents answered questions using an online survey tool.

In addition to checking your résumé, nearly 3 out of 4 hiring managers and recruiters check candidates’ social profiles, even if they are not provided on your résumé, with some of the content garnering negative reactions. Content that recruiters especially frown on includes references to using illegal drugs (78% negative) and posts of a sexual nature (67% negative). Profanity in posts and tweets garnered a 61 % negative reaction, and almost half (47%) reacted negatively to posts about alcohol consumption. Worse than drinking, grammar or spelling mistakes on social profiles saw a 54% negative reaction from respondents. However, recruiters and hiring managers tend to be neutral in their reactions to political opinions (62% neutral) and religious posts (53% neutral).

Survey Questions and Answers: The Presence of Error

In order to conceptualize the error associated with answers to survey questions, it important to address what a survey is designed to measure. At its most basic, surveys try to measure two items: objective facts and subjective states. Objective facts include the number of times a person has been arrested, whether or not a person has used marijuana in the past week, and whether or not a person has been the victim of theft in the past 6 months. Subjective states include a person’s level of attachment to his parents and a person’s viewpoint on the primary purpose of prisons.

As will be discussed in the next section, the way we assess the answers to a survey question is to measure how well they correspond to the truth. If you are asking survey respondents about objective facts (e.g., how many times have you been arrested?), you can obtain independent information (e.g., criminal history information from a statewide database) against which to evaluate the answers to the survey question. Unfortunately, there is no objective way to verify or evaluate a person’s report about a subjective state (e.g., viewpoint on the primary purpose of prisons). There is no way to independently assess whether the person is telling the truth, which can cause error in the survey results.

A defining characteristic of survey research is that answers to the survey questions are used as the measurements of the concepts and variables in the study. The extent to which those answers are good measures of the concepts and variables is obviously a critical dimension of the quality of survey results. The extent to which the answers are not good measures creates survey error. Survey error comes from several areas, including misunderstanding the question, not having the information needed to answer, and distorting answers in order to look good, to name a few. 18 For example, respondents typically underreport how much alcohol they drink and overstate their income. Survey results of alcohol consumption and income are likely to be biased (i.e., systematically different from the true scores). The issue is that to the extent that answers are impacted by factors other than the facts on which the answer should be based, there is error in the answer. 19 The next section discusses the quality of the survey measurement and related error by discussing reliability and validity.

Survey Measurement Quality: Validity and Reliability

Assessing the quality (i.e., accuracy and consistency) of a survey measurement is a critical part of the research process. A survey researcher can spend days developing what he views as an excellent survey, but if the survey questions and responses are not accurate and consistent, the results of the study are questionable. Accurate and consistent survey questions and responses are essential to getting accurate results and making conclusions that reflect the true reality of the topic being studied. Specifically, the two key components of survey quality are reliability and validity. In order to be a high-quality measurement, survey questions and responses must be both reliable and valid. Reliability is necessary for quality measurement, but not sufficient. A quality measure must also be valid. Similarly, just because a measure is valid doesn’t necessarily mean it is reliable, and validity means little if the measure used is not reliable.

Validity addresses the accuracy of the measurement and refers to the extent to which researchers measure what they planned to measure. Validity refers to the accuracy and trustworthiness of survey instruments, data, and findings. The question asked when assessing validity is: “Are the survey questions measuring what they are intended to measure?”

Reliability addresses the consistency of a measurement and refers to whether or not you get the same answer if you use an instrument to measure something more than once. For example, if a police radar gun is used to measure the speed of two vehicles going the same speed and the same results are obtained, then the radar gun is reliable. Similarly, if a breathalyzer is administered to two individuals with the same blood alcohol level and the breathalyzer gets the same results each time, then the breathalyzer is reliable. In these cases, the radar gun is a reliable measure of the speed of automobiles, and the breathalyzer is a reliable measure of blood alcohol content (i.e., level of intoxication).

As previously discussed, surveys generally assess two different types of information: objective facts and subjective states. Generally, it is easier to accurately and consistently measure objective facts than subjective states. For example, it is easier to accurately and consistently measure a survey respondent’s gender, race, ethnicity, and education level (i.e., objective facts) than it is to measure a survey respondent’s opinion (i.e., subjective state) regarding the level and quality of police service provided to the community. The extent to which a survey is unreliable and invalid creates measurement error, which is a persistent problem in criminal justice and criminology research. One of the major sources of error in research studies is poor quality of the measurements.

Both validity and reliability must be present for high-quality research results. This section will address ways to measure and increase the validity and reliability of a measurement. It is important to note that validity and reliability do not just apply to survey research but to measurement completed in all research designs, including experimental designs (Chapter 5), field research (Chapter 6), case studies (Chapter 6), secondary data analysis (Chapter 7), and content analysis (Chapter 7). Since this is the first chapter to discuss measurement, a comprehensive review of validity and reliability is presented here. As you proceed through the remainder of this book, keep in mind that validity and reliability apply to all types of measurement that are completed in research studies.

Assessing Validity

Validity addresses the accuracy of the measurement and refers to the extent to which the survey questions and responses measure what they were supposed to measure. In other words, are the survey questions and responses good indicators of what the researchers are trying to study? Stated another way, are the survey questions and responses measuring what they are intended to measure? There are numerous ways to assess the validity of measures. The ways range from simply asking if the measures seem like logical and common sense ways to measure the concepts and variables (i.e., face validity) to the complexity of determining whether the measures fit the theoretical constructs of the concept being measured (i.e., construct validity). Even though researchers may use sophisticated ways to assess the validity of their measures, concerns about the validity of the measure may still persist because of the complexity of the concepts studied by criminal justice and criminology researchers. In addition, validity is never proven; instead, invalidity is reduced, and when this occurs researchers express greater confidence in their data and the results of their study. There are several ways of assessing if the measures used in a study are valid: face, content, criterion-related, and construct validity. The different means of assessing validity vary by complexity and subjectivity.

Face validity, the simplest and most subjective means to measure validity, assesses whether the survey questions are a logical and common sense way to measure the concept. Basically, face validity involves an assessment of the survey questions to see if on “face value” the questions seem to be measuring the concepts and variables they are supposed to be measuring. Face validity answers the simple question, “Do the survey questions look like they measure what they are supposed to be measuring?” For example, if a researcher is measuring fear of crime, does the survey instrument ask questions about specific types of crime and the level of fear for each? If so, the survey has face validity.

Face validity is sometimes developed based on establishing a consensus among researchers that the measure is a valid way to measure the concept. For example, if researchers generally agree that asking someone “How many times have you been arrested?” is a valid measure of prior arrests, then, until proven otherwise, the question is a valid measure of prior arrests. However, because face validity is entirely a matter of judgment, there can be great disagreement among researchers and consumers of research about whether a survey question has face validity.

Content validity is subjective, just like face validity, but is somewhat more complex than face validity. It occurs when the survey questions measure the full breadth and depth of the concept being studied. For example, let’s say that when you have completed Chapters 1–4 of this book, you are scheduled to take an exam over Chapters 1–4. What if when you take the exam, your professor only asks questions about sampling, which is covered in Chapter 3? You would probably argue that the test was not fair and was not a good measure of your knowledge of Chapters 1–4 because it only covered Chapter 3. What you are stating is that the test, as a measurement of knowledge of the material covered in Chapters 1–4, lacks content validity. In order to have content validity, the test should have included questions over Chapters 1, 2, 3, and 4.

As another example, there exists a traditional measure of fear of crime that lacks content validity. Traditionally, fear of crime was measured in surveys based on the response to the question, “How safe do you feel walking alone in your neighborhood at night?” This question lacks content validity because it does not measure the full breadth and depth of someone’s fear of crime. It is basically limited to fear of street crimes like robbery and assault, but does not include crimes that people fear but occur in their residences, such as sexual assault, family violence, burglary, and theft. Also, the question is limited to your neighborhood, whereas you may have a high level of fear of crime but it is due to getting off late at night from your job and having to walk through a deserted parking garage to get to your car. Furthermore, the question is time dependent, only asking about fear of crime at night.

When it comes to criminal justice and criminology research, content validity is difficult to obtain because researchers are trying to measure complex concepts. When studying criminal offending, it is difficult to ask about all types of crimes; there are just too many. Similarly, when studying drug use, it is difficult to ask about all types of drugs. However, just as discussed with face validity, consensus among researchers is used to determine content validity.

Criterion-related validity is more complex and less subjective than face and content validity. Criterion-related validity is assessed by determining the strength of the relationship between the responses to the survey and another measurement, the criterion, to which it should be related if the measurement is valid. For example, let’s say that you want to create a measure of self-esteem. As previously mentioned in this chapter, the 10-question measure of self-esteem created by Rosenberg in the 1960s is a valid measure of self-esteem. You create a 16-question measure of self-esteem but will want to determine if the measure is valid. You can give a sample of people a survey that includes both the Rosenberg questions and your new questions about self-esteem. You can then determine the relationship between the responses to your measure of self-esteem in comparison to Rosenberg’s questions, the criteria. If individuals that have high self-esteem according to Rosenberg’s measure consistently have high self-esteem according to your new measure of self-esteem, then your measure has criterion-related validity. If individuals have high self-esteem according to Rosenberg’s measure but consistently have low self-esteem according to your new measure, then your measurement of self-esteem lacks criterion-related validity.

As another example, you may be interested in pursuing law school after completing your bachelor’s degree. Admission committees at law schools use the Law School Admission Test (LSAT) to predict success in law school. They prefer to admit students with high LSAT scores because they believe those are the students most likely to do well in law school. If there is a strong relationship between the LSAT score and success in law school as measured by law school grades, then the LSAT has high criterion-related validity. The LSAT is related to something that it should be related to, in this case, law school grades.

There are two types of criterion-related validity: concurrent and predictive validity. A measurement has high criterion-related validity, more specifically high concurrent validity, when there is a strong relationship between a new measure and a measure that is known to be valid. The example above regarding Rosenberg and a new measure of self-esteem is an example of concurrent validity. Concurrent validity occurs when an experimental measure (e.g., your new self-esteem measurement) and a valid measure (e.g., Rosenberg’s self-esteem scale) measure the same thing at the same time (i.e., concurrently).

Concurrent validity is also assessed when a response to a survey item is compared to the same information from a different data source. For example, let’s say your professor asks your current grade point average on a self-administered survey. Then, your professor uses your university’s information system to obtain your official GPA. The more these two numbers correspond (i.e., your stated GPA and your official GPA), the higher the concurrent validity of the survey measurement of GPA. In criminal justice and criminology research, this process can occur to validate survey information that is also stored by criminal justice agencies such as prior arrests, convictions, and incarcerations, to name a few.

Another way to assess concurrent validity is to use what is sometimes referred to as the known group comparison technique. Let’s say that you want to develop a measure of religiosity because you want to study the impact of religious beliefs on delinquent activities. Although valid measures of religiosity exist, you want to develop your own measure. After you have developed the survey questions, you administer the survey to members of the clergy and members of Atheist Alliance International (AAI). If your survey measure of religiosity shows that the members of the clergy have high levels of religiosity and members of AAI have low levels of religiosity, then you have established the concurrent validity of your measure. The items in your survey measurement are measuring what they were intended to measure (i.e., level of religious beliefs). If the measurement doesn’t differentiate the level of religious beliefs between members of the clergy and AAI, then there is something wrong with your survey questions.

Another type of criterion-related validity assesses how accurately a measurement predicts some future, rather than current, outcome similar to the example previously discussed regarding the LSAT as a prediction of law school success. This type of validity is referred to as predictive validity. Let’s say that you develop a set of survey items that is designed to determine if someone will be a successful police officer. A local police department decides to assist you with your study and allows you to issue your survey to the current police recruit academy class of 175 cadets. You track these police recruits for the next three years after completion of the police academy and field training program. If the individuals who scored highest on your survey while in the academy are also the most likely to have received promotions, excellent performance evaluation reviews, and fewer disciplinary infractions, then your measure has predictive validity. Police departments can now use this measurement tool to screen potential police officers with confidence in its ability to predict future success.

We frequently use measures in the criminal justice system to make decisions, including whether a crime can be solved, whether someone should be let out of jail on bail, whether a person should be released from prison on parole, and whether a person should be placed on a general probation caseload or a specialized caseload. These measures are only as good as their level of predictive validity; ability to predict some future outcome. If they have predictive validity, then we can have confidence in the predictive accuracy of the measurement.

Another way of assessing the accuracy of a measure is to determine its construct validity. Similar to criterion-related validity, construct validity is more complex and less subjective than face and content validity. Construct validity assesses the extent to which a particular measure relates to other measures consistent with theoretically derived hypotheses concerning the concepts/variables that are being measured. Since construct validity is based on theory, it is favored by researchers.

For example, prior research has established a relationship between self-esteem and delinquency. Juveniles with high self-esteem are less likely to commit delinquent acts than juveniles with low self-esteem. You want to conduct a research study to test the following hypothesis: There is a relationship between self-esteem and delinquency. As presented in prior examples, let’s say that you want to create your own measure of self-esteem instead of using Rosenberg’s validated measure. You create your new 16-item measure of self-esteem and use a prior validated measure of delinquency. You administer the survey to the 8th graders in the middle school closest to your residence. The results show that those with high levels of self-esteem are less likely to commit acts of delinquency and those with low levels of self-esteem are more likely to commit acts of delinquency. This is exactly what was expected based on your theoretically derived hypothesis regarding the relationship between self-esteem and delinquency. Therefore, your new measure of self-esteem has construct validity. If the survey results would have shown that those most likely to be delinquent also have the highest levels of self-esteem, then your measurement of self-esteem would lack construct validity and the results of the study would be suspect. As depicted in the example, construct validity is based on actual research results and is not obtained until after the data has been collected and analyzed. Other researchers can begin to use your measure of self-esteem because you have shown that it has construct validity.

FIGURE 4.1 | Mechanisms for Assessing Validity

Increasing Validity

Now that we have presented the main approaches to assessing validity, we turn our attention to discussing ways to increase the validity of a survey instrument. As previously discussed, it is certainly reasonable for a researcher to use measures that have already been established as valid. If you are studying victimization on your college campus, it is certainly reasonable to develop your survey based on the National Crime Victimization Survey since the validity of this measurement has already been established. Even though a researcher may be using a previously established measure, there are validity issues that must be addressed in every research study. This section will discuss the major validity issues and the suggested means to overcome them, thereby increasing the validity of a research study. As an educated consumer of research, you should be aware of these issues as you review research reports. One significant validity issue with survey research is whether respondents are telling the truth.

Are the Respondents Telling the Truth? Aiding Respondent Recall Recall that validity is addressing the accuracy of the measurement. Survey responses are usually accurate and truthful when the survey questions ask about a behavior or incident that is easily remembered. Most offenders can easily remember how many times they have been arrested in the past month, but it is more difficult for a drug abuser to remember how many times they have used illicit drugs in the past month. It is likely that the number of arrests in the past month will be a more accurate response than the number of times an abuser has used drugs in the past month.

It is clear that a respondent’s ability to accurately recall events declines over time and accurately recalling events typically takes more time than the few seconds that a respondent has to answer a survey question. As discussed in the chapter opening case study, a major source of error of the National Crime Victimization Survey (NCVS) includes the inability of the respondents to recall in detail the crimes that occurred during the 6 months prior to the interview. The NCVS uses a 6-month reference period. Respondents are asked to report victimization experiences occurring in the last 6 months. Generally, respondents are able to recall more accurately an event that occurred within 3 months of the interview rather than one that occurred within 6 months. Research indicates that assault is recalled with the least accuracy of any crime measured by the NCVS. Recall problems may result in an underestimate of the actual rate of assault.

Survey researchers recognize that memory is less trustworthy than once assumed. In response, strategies have been developed to maximize respondent recall. Survey researchers should be as specific and recent as possible when asking about past events. You can improve accuracy by making the recall period for the survey questions recent. The problem of forgetting becomes increasingly serious as the respondent is asked to recall events stretching over a longer period of time. In addition, the researcher should clearly specify the behavior asked about. The complexity of respondent recall does not mean that survey researchers cannot ask about past events. Instead, survey researchers need to customize questions to shorten the recall period and interpret results cautiously as the recall period grows larger.

In addition, many respondents will telescope their responses. Telescoping occurs when a respondent brings behaviors and actions that occurred outside the recall period into the recall period. For example, let’s say that a person was assaulted 8 months ago and has not been assaulted since that time. The person is answering a survey question that asks “How many times have you been assaulted in the past 6 months?” and the survey respondent answers “one time.” The respondent has telescoped the victimization into the 6-month time frame even though it occurred 8 months ago and was outside the time frame provided by the researcher. Telescoping responses leads to overreporting of recent events and underreporting of distant events.

Are the Respondents Telling the Truth? Asking Embarrassing and Stigmatizing Questions People have a tendency to underreport embarrassing or stigmatizing events (e.g., hitting their children, being a sexual assault victim, and lying on their tax return). Criminal justice research asks about many potentially embarrassing and stigmatizing topics such as criminal offending, victimization, drug use, alcohol consumption, mental illness, and prison rape, to name a few, thus making truthful reporting a challenge in criminal justice research. Respondents may be embarrassed or afraid to give truthful answers and thus, underreport behaviors or attitudes they wish to hide from others. As an educated consumer of research, you need to be aware of and assess the strategies a researcher uses to increase the honesty of survey responses.

Dishonest answers can be difficult to detect, so what strategies exist to increase the honesty and therefore, accuracy of survey responses? First, researchers should inform respondents that their responses are, at a minimum, confidential, and if possible, anonymous. Anonymity infers that the survey respondent’s identity will not be known by the researcher. For example, if a professor hands out a survey in her class to the 150 people in attendance, asks them to respond to the survey and hand them to the end of the seating row where they are picked up by the professor, the surveys are anonymous. The professor does not know who filled out a particular survey. Anonymously collecting survey information, especially if it is of a sensitive or criminal nature, is one of the simplest ways to increase the honesty of responses. Respondents will be more honest regarding sensitive or embarrassing topics when they know their responses cannot be used to harm or embarrass them.

Confidentiality infers that although the researcher may be able to link the information given on a survey or during an interview to the particular research participant who gave it, this link will not be released to anyone else besides members of the research team. For example, a researcher who does face-to-face interviews will certainly know the identity of the respondent but promises to keep the responses in confidence.

Second, survey researchers should try to ensure that sensitive questions are asked in a nonjudgmental way. The researcher should minimize any sense of judgment and maximize the importance of accuracy. Careful attention should be paid to the wording of the introduction and the question to ensure that nothing negative about the behavior is implied to the respondent. The researcher needs to be very careful about any cues respondents are receiving about how their answers will be interpreted. These questions must be asked with great care and great empathy.

Third, the survey researcher should use a self-administered data collection procedure (discussed later in this chapter). It is generally agreed that having respondents answer questions in a self-administered form, on paper or directly into a computer, rather than having an interviewer ask the questions, will produce more honest answers. For surveys dealing with sensitive topics, a mail survey, Internet survey, or group administration survey should be considered. A face-to-face interview can also include some self-administered questions where a respondent is given a set of sensitive questions to answer in a booklet or directly into a computer. A similar strategy can be used with telephone surveys where answers to sensitive questions are entered directly into a computer using the touch-tone feature on the telephone.

Fourth, respondents feel more comfortable answering potentially sensitive questions when the responses are provided as categorical ranges rather than as a specific number. For example, a researcher may get a more honest answer if instead of asking, “What is your annual income?” the researcher provides the respondent with ranges of income from which to choose a response (e.g., less than $20,000, $20,000–$35,000). A respondent’s age can be collected with categorical ranges, instead of a specific age, as well. Especially with demographic characteristics, the more sensitive the question, the better it is to use response categories instead of specific numeric values.

Fifth, the survey researcher can use a randomized response technique (RRT), which allows the researcher to ask about illegal and sensitive behaviors and obtain accurate estimates of the frequency of the behavior. Here’s how it works. Let’s say you are interested in the frequency of excessive force used by correctional officers. You are conducting face-to-face interviews with a sample of correctional officers at a prison unit. Your interviewers define excessive force for each respondent. But instead of your interviewers asking correctional officers the question directly, they give each respondent a randomization device, such as a coin. They ask the respondent to flip the coin. They tell the respondent not to tell them the results of the coin toss. Then they instruct the respondent as follows: “Say Yes if either of the following is true: Your coin came up heads, or you have used excessive force against an inmate in the last month.”

If the respondent says yes, the interviewer does not know which question is being answered. Did the correctional officer say yes because the coin landed on heads or because the officer has used excessive force against an inmate in the last month? If this procedure was repeated with 100 respondents, based on probability theory, you would expect 50 out of 100 to say “Yes,” simply because their coin toss came up heads. Therefore, if you find that 68% of the respondents say “Yes” to this question, you estimate that the 18% above the expected 50% represents correctional officers who used excessive force against inmates in the past month. Thus your finding is that 18% of correctional officers use excessive force against inmates in any given month.

Sixth, although this will be discussed in a later section of this chapter, realize that researchers can use one of the reliability tests, such as split-half reliability, to detect untruthful responses. Researchers ask the same or similar questions in different sections of a survey to determine if respondents were truthful in their responses. Surveys in which the truthfulness of the respondent is questioned should be discarded from the study prior to analysis.

Are the Respondents Telling the Truth? Social Desirability People have a tendency to overreport something if it makes them look good (e.g., willingness to intervene in a crime to protect someone). Survey researchers call this overreporting social desirability bias. Social desirability bias occurs when respondents provide answers to survey questions that do not necessarily reflect the respondent’s beliefs but that reflect social norms. If social desirability bias is widespread, it can have a significant impact on the results of the study.

One of the best examples of social desirability bias is LaPiere’s 20 study of attitudes and behaviors towards hotel and restaurant service of Chinese people. Beginning in 1930 and continuing for two years thereafter, LaPiere traveled across the United States in an automobile with a young Chinese student and his wife. The early 1930s was a time when the general attitude of Americans toward Chinese residents was negative and was well documented in several social distance studies completed at the time. There was a national negative stereotype of Chinese citizens at the time, and the social norms of the time were to react negatively to individuals of Chinese descent. Discrimination against Chinese residents was common as well.

LaPiere discovered substantial differences between the stated attitude of survey respondents and actual behavior. During their extensive travels, the Chinese couple was received at 66 hotels, auto camps, and “tourist homes,” and was only refused service at one. Similarly, they were provided service in 184 restaurants and cafes throughout the United States and were never refused service. To assess differences between stated attitude and behavior, LaPiere completed a mailed survey six months after departure from 22 of the hotels and 43 of the restaurants they visited and asked “Will you accept members of the Chinese race as guests in your establishment?” To his bewilderment, 91% of the hotels and 93% of the restaurants answered “No,” even though he had a personal experience with each of the establishments where all but one provided service to the Chinese couple. Simply stated, he discovered what people say is not always what they do.

LaPiere’s study certainly illustrates the difference that occurs sometimes between stated attitudes on surveys and actual behaviors. Just because someone says they would or would not do something on a survey, does not necessarily mean that will relate to the respondents’ actions. In this case, LaPiere observed substantial discrepancies between attitudes and behaviors.

This study is also a reflection of social desirability bias. At the time, the social norms dictated a negative reaction to individuals of Chinese descent. When presented with a question about whether they would provide service to someone of Chinese descent, almost all of the respondents reflected the social norms at the time and responded “no.” As stated above, social desirability bias occurs when respondents provide answers to survey questions that do not necessarily reflect the respondent’s beliefs but instead reflect social norms, and that is certainly the case in LaPiere’s study.

In order to detect social desirability, some survey researchers include the same or similar questions in different sections of a survey to determine if respondents were truthful in their responses, as previously mentioned. Researchers can also build in questions that are likely to identify respondents who are giving socially desirable answers. For example, the researcher may ask survey questions that include words such as “always” and “never.” People rarely “always” or “never” do or feel something, so respondents who routinely select these responses are probably depicting social desirability bias. Another way to detect social desirability bias is to include unlikely choices in the closed-ended survey questions. Surveys in which the truthfulness of the respondent is questioned should be discarded from the study prior to analysis.

CLASSICS IN CJ RESEARCH

The Relationship Between Correctional Officers’ Race and their Attitudes

Research Study 21

Historically, correctional officers were white, uneducated, conservative males who lived in the rural areas where most prisons were located. Beginning in the 1950s, prison populations started to change as more urban blacks were sent to prison. Racial tensions between inmates and guards escalated. Black inmates had not only become more prevalent in number, they stood as a unified group against the guards who perceived them as a greater threat compared to the white inmates. Many reformers advocated increasing the number of black prison guards as a way to reduce conflict and antagonism between guards and inmates. The assumption was that black guards would be able to relate better to the black inmates because they shared similar backgrounds. Whether or not the attitudes held by black prison guards toward inmates were any different from those held by white guards remained an untested assumption until the late 1970s when James Jacobs and Lawrence Kraft conducted an empirical test of correctional officers’ attitudes.

The authors tested the following five hypotheses:

1. Black guards have more sympathetic attitudes toward inmates compared to white guards.

2. Black guards are more supportive of rehabilitation compared to white guards.

3. Black guards convey less support for their superiors compared to white guards.

4. Black guards are less committed to institutional goals compared to white guards.

5. Black guards are less committed to their occupations compared to white guards.

Is there a relationship between correctional officers’ race and their attitudes toward inmates, correctional goals, administrators, and correctional officer roles?

The authors administered a survey to a sample of 252 guards from Stateville and Joliet prisons who were attending in-service training. Both prisons were maximum-security facilities within seven miles of each other. Three-fourths of the inmate populations in both prisons were black, and most of the prisoners came from the Chicago area (the prisons were 35 miles southwest of the city). At the time the survey was administered, 12% of the correctional officers employed by the Illinois Department of Corrections were black and most of them were working at Stateville and Joliet.

The survey was administered to guards while they attended in-service training at the Correctional Academy between the summer of 1974 and fall of 1975; 165 white guards and 66 black guards completed the survey; 21 guards were excluded from the sample because they did not indicate their race and/or rank on the questionnaire. Prison guards were group-administered an anonymous survey and asked to respond to a series of questions that consisted of both closed-ended and open-ended questions.

Specifically, inmate orientation consisted of a 10-item measure, one question open-ended and the other nine closed-ended, which asked questions such as, “Inmates try to take advantage of officers whenever they can.” Job orientation consisted of a 7-item measure, which asked all closed-ended questions such as, “Although understanding may be important in helping inmates, what is really needed is strictness.” Staff orientation was a 3-item measure, all closed-ended questions such as, “When I began, the veterans were friendly and helped me learn.” System orientation was a 5-item measure, one question open-ended and the other four closed-ended, such as, “Why are there so many members of minority groups in prison?” Job commitment was a 9-item measure with all closed-ended questions such as, “Thinking ahead five years from now, do you think you will still be a correctional officer?”

The results of the study for each hypothesis are presented below.

1. Hypothesis: Black guards have more sympathetic attitudes toward inmates compared to white guards.

Study finding: Black prison guards did not express attitudes that were more sympathetic toward inmates compared to white guards. The responses of black guards to several questions actually revealed less sympathy. The hypothesis was rejected.

2. Hypothesis: Black guards are more supportive of rehabilitation compared to white guards.

Study finding: Black and white prison guards both indicated that rehabilitation was the primary purpose of prison, but when asked what the “primary purpose of prison should be,” more black guards than white guards chose punishment. The hypothesis was rejected.

3. Hypothesis: Black guards convey less support for their superiors compared to white guards.

Study finding: Black prison guards responded more favorably to their superiors than white guards even though most of the prison administrators and supervisors at the two prisons were white. The hypothesis was rejected.

4. Hypothesis: Black guards are less committed to institutional goals compared to white guards.

Study finding: Black officers do not support a more relaxed correctional process. The majority of both races disagreed with the statement that “correctional officers should be rough with inmates occasionally to let them know who is boss,” but blacks more often gave approval to the statement. The hypothesis was rejected.

5. Hypothesis: Black guards are less committed to their occupations compared to white guards.

Study finding: The responses of both black and white guards indicated a strong sense of institutional commitment. Two-thirds of the guards sampled responded that they planned to be working at the prison for at least the next five years. Many of the guards expressed a preference for their current occupation over other alternative options such as private security and police work. The hypothesis was rejected.

Overall, the conclusion was there were no consistent differences by race in guards’ attitudes toward prisoners, staff, correctional goals, or their occupation. According to the study authors, “There is nothing in these responses to suggest that black guards treat inmates with greater respect or sensitivity. They do not hold more rehabilitative views. Nor have they aligned themselves with the inmates against the administration.” 22

Jacobs and Kraft recognized one of the potential validity issues with their questions was social desirability. Guards’ responses may have been self-serving. Respondents may have provided “socially desirable” answers. The guards may have responded to the questions in terms of how they felt they should respond, not based on their own personal feelings. The extent that social desirability influenced the guards’ responses was not known. As discussed in this chapter, social desirability is a source of measurement error that threatens the validity of a researcher’s measures. Survey questions are valid only if they correctly assess the phenomenon under study; in this case, the guards’ attitudes toward prisoners, staff, correctional goals, and their occupation.

Another limitation of this study is that the sample of prison guards was not randomly selected. A non-probability sampling method was used to draw the sample. As discussed in Chapter 3, since a non-probability sampling method was used, in this case purposive sampling, the sample of prison guards selected for this study are likely not representative of a larger population. Jacobs and Kraft purposely chose prison guards from Joliet and Stateville prisons who were completing in-service training at the state’s Correctional Academy because most of the black prison guards in Illinois worked at these two prison facilities. However, since non-probability sampling was used, the sample may not even be representative of all prison guards in these two prison facilities and is even less likely to be representative of prison guards in Illinois or the United States. Therefore, the results may not be generalizable to these other populations.

Reliability issues may have arose with the wording of some of the survey questions as well. For example, in the inmate orientation scale, Jacobs and Kraft asked “in your opinion, when just considered as people, how similar are guards and prisoners?” The question was supposed to mean how similar is the prison guard answering the question to the inmates. Respondents may have understood the question differently with some respondents having answered in terms of other guards or most guards, not themselves personally. Therefore, it may be misleading to consider this item an indicator of perceived distance between self-as-guard and prisoners.

Jacobs and Kraft recommended that recruitment of minority guards was itself an important societal goal, because it expanded job opportunities in an area where minorities had been traditionally excluded. For this reason alone increasing the proportion of black guards is socially justifiable as well as legally compelled. The study discovered, though, that we should not unquestioningly accept the belief that this change in demographic composition of the work force will automatically have a major impact on the atmosphere of the prison.

Neither black nor white guards displayed attitudes that indicated they were more “inmate oriented.” The similarities suggested that the attitudes might have been formed as a result of their socialization into the prison environment. Just as police officers develop “working personalities” that influence their interactions with the public, guards display certain attitudes and behaviors toward inmates. Numerous studies published after Jacobs and Kraft have identified various organizational influences on guards’ attitudes.

The number of black correctional officers employed in prisons across the United States has increased in the years since Jacobs and Kraft published their study. Jacobs and Kraft were the first to explore the influence of race on correctional officer attitudes and, since then, numerous studies of correctional officers’ attitudes have been published. The debate over whether or not hiring more black correctional officers will ease racial tensions between officers and inmates continues due to contradictory research findings, but according to Jurik, 23 it would be unrealistic to expect significant improvements to the prison environment by simply hiring officers with particular demographic characteristics.

Increasing Validity: Using Scales and Indexes

Recall that validity addresses the accuracy of the measurement and refers to the extent to which the survey questions and responses measure what they are supposed to measure. One way of increasing the validity of your survey questions and responses is to ask multiple questions to measure the same concept, especially if the concept is complex. You do not need to ask five different questions to measure a respondent’s age. But, what about measuring parental attachment? You could have one question in your survey that asks, “What is your level of parental attachment?” but there are validity issues with asking just one question to measure a complex concept. As an alternative, you could ask the following 11 questions to measure parental attachment:

How often would you say that…

1. You get along well with your mother?

2. You feel that you can really trust your mother?

3. Your mother does not understand you?

4. Your mother is too demanding?

5. You really enjoy your mother?

6. You have a lot of respect for your mother?

7. Your mother interferes with your activities?

8. You think your mother is terrific?

9. You feel very angry toward your mother?

10. You feel violent toward your mother?

11. You feel proud of your mother?

The response categories for each question can be almost never, sometimes, most of the time, almost always.

Which is a better, more valid measure? Certainly, the measurement of parental attachment that asks 11 different questions is a higher quality measurement than just asking one question. In fact, the questions above were used by Bjerregaard and Smith 24 in their assessment of gender differences in gang participation. Their measurement of parental attachment is an adaptation of Hudson’s 25 Child ’ s Attitude Toward Mother (Father) Index , a well-standardized and validated index in the family assessment literature. By asking multiple questions to measure a complex concept, the researcher is able to even out response idiosyncrasies and improve the validity of the measurement.

Therefore, to measure a complex concept, researchers construct scales and indexes. These words are often used interchangeably in research methods, but they are actually different, so we will make some distinctions between the two. An index is a set of items that measure some underlying concept. Creating an index involves developing a set of items that, taken together, are indicators of the underlying concept you are attempting to measure. For example, every weekday the Dow Jones Industrial Index (DJJI) is reported on the nightly news. The DJII is a set of numerous stocks that are combined to create a single number. Rosenberg’s self-esteem measure, which was previously discussed, is an index. It uses 10 questions to measure the underlying concept of self-esteem. Similarly, the 11 parental attachment questions listed above form an index. The 11 items, taken together, are indicators of the underlying concept of parental attachment. So, when multiple questions are used to measure a complex concept, an index has been created.

What does a researcher do with the responses to the individual questions that make up the index? The researcher calculates an overall index score by summing the responses to all of the questions. The score serves as the indicator of your level of the concept being measured. Let’s use the parental attachment index as an example. Each of the 11 questions had four possible response categories: almost never, sometimes, most of the time, almost always. Almost never is coded as 1, sometimes is coded as 2, most of the time is coded as 3, and almost always is coded as 4. Each respondent’s answers to all questions are summed to get a single number that represents the respondent’s level of parental attachment. The scores range from 11 (i.e., answered each question as “almost never”) to 44 (i.e., answered each question as “almost always”) with a higher score representing a higher level of parental attachment. One respondent may have a score of 22 on the parental attachment index while another respondent may have a score of 37, the latter having a higher level of parental attachment than the former.

If you tried to do the scoring of the parental attachment index on your own, you may have run into a problem. Some of the questions are positive (e.g., you get along with your mother) and some are negative (e.g., your mother is too demanding). A response of “almost always” on the first question in the previous sentence demonstrates a high level of parental attachment, but a response of “almost always” on the second question in the previous sentence demonstrates a low level of parental attachment. How does a researcher deal with this issue? The answer is the researcher reverse codes the negative statements. Instead of using the 1-4 coding scheme discussed in the last paragraph, the researcher reverses the code for the negative items. In this case, almost never is coded as 4, sometimes is coded as 3, most of the time is coded as 2, and almost always is coded as 1. Therefore, when a respondent answers “almost never” to the “your mother is too demanding” question, the answer is coded as a 4, which demonstrates a high level of parental attachment.

A scale is a set of questions that are ordered in some sequence. The researcher is seeking a pattern from the respondent’s answers to the set of questions, rather than a simple summation of the individual question responses, as happens with indexes. For example, Guttman scales are used by survey researchers and are ordered in such a way that agreement with a particular item indicates agreement with all the items that come earlier in the ordered set of statements. Consider the following Guttman Scale regarding immigration.

Place a check next to each statement you agree with

—— I would be comfortable with the United States expanding immigration.

—— I would be comfortable with new immigrants moving into my city.

—— I would be comfortable with new immigrants moving onto my block.

—— I would be comfortable with new immigrants moving next door to me.

—— I would be comfortable with a new immigrant dating my child.

—— I would be comfortable with a new immigrant marrying my child.

The items are ordered so that agreement with the fourth item would also indicate a strong likelihood of agreement with the first three statements. If the respondent is not comfortable with new immigrants moving into his city, then he is definitely not going to be comfortable with new immigrants moving next door.

You have probably also heard someone say “On a scale of 1 to 10 …” For example, it is common for health care providers to ask patients to rate their pain on a scale of 1 to 10. You have probably also heard someone mention the term Likert scale. Scales are used to measure the intensity of your response. Just like the pain scale measuring the intensity of your pain, a Likert scale measures the intensity of your response. A Likert scale measures the intensity of your preference or opinion and is typically measured on a five-point scale, such as “strongly agree,” “agree,” “neither agree nor disagree,” “disagree,” and “strongly disagree.” Responses to several questions using a Likert scale are often combined to form an index. Now, you probably understand why the words “index” and “scale” are used interchangeably by many people. The word “scale” is used more frequently than “index.” Although there are differences between the two terms, the key is to understand how scales and indexes are used in criminal justice and criminology research rather than worrying about whether the research study you are reviewing used a scale or an index. Also, remember that complex concepts should be measured by numerous questions.

Assessing Reliability

In our everyday lives, we use the terms reliable and unreliable to describe our cars, friends, and technology, among others. If your car starts every time you are ready to leave, then it is reliable. If your friend never helps you when you request assistance, then he or she is considered unreliable. If your cell phone routinely drops calls, then it is unreliable. Basically, reliable, as it has been used in these examples, involves consistency. For example, does your car consistently start? As another example, if you measure your weight on your bathroom scale five times in a row and receive the same result, then the scale is a reliable measure of weight. The scale has consistently measured your weight. Stepping on your bathroom scale three times in a row and getting readings of 108, 212, and 157 pounds is inconsistent, unreliable, and as a result, not very useful.

We also use the term to describe items in criminal justice as previously mentioned. For example, if a police radar gun is used to measure the speed of two vehicles going the same speed and the same results are obtained, then the radar gun is reliable. Similarly, if a breathalyzer is administered to two individuals with the same blood alcohol level and the breathalyzer gets the same results each time, then the breathalyzer is reliable. In these cases, the radar gun is a reliable measure of the speed of automobiles, and the breathalyzer is a reliable measure of blood alcohol content (i.e., level of intoxication).

Researchers use the term in roughly the same way to describe the measures they use in their research. Reliability, from a research standpoint, addresses the consistency of a measurement. It refers to whether or not you get the same result if you use an instrument to measure something more than once. It is the researcher’s expectation that there will not be different results each time the measure is used, assuming that nothing has changed in what is being measured. When a researcher obtains different results on different measurements, then there is random error in the measurement process and the results must be evaluated with caution.

Just as with validity, there are some common ways for researchers to assess the reliability of their measures, but the assessment used depends on which aspect of reliability they are interested in. There are three different aspects of reliability that are of interest to criminal justice researchers: 1) reliability over time; 2) reliability over raters or observers; and, 3) reliability over items. The survey researcher could administer the same survey to the same respondents twice to assess consistency. This is known as test-retest reliability and is used when the researcher is interested in reliability over time. The survey researcher could have two different persons gather observations and assess the consistency between the two. This is known as interrater reliability and is used when the researcher is interested in reliability over raters or observers. Frequently, researchers assess reliability of their measures by dividing the survey into two halves and comparing the results of each half. This is known as split-half reliability and is used when the researcher is interested in reliability over items. Each of the three common ways to assess the reliability of a survey instrument is discussed in the next sections.

Test-Retest Reliability—Reliability over Time If a researcher is interested in the reliability of a survey instrument over time, she can use test-retest reliability. Utilizing test-retest reliability, a measurement is reliable over time if the same measurement is repeated using the same subjects and yields the same results. Let’s use the Scholastic Aptitude Test (SAT) as an example. If we administer the SAT to 100 high school seniors tomorrow and then administered the SAT to the same 100 seniors two days later, we would expect each student to get similar results on each test (assuming learning did not take place from the first administration to the second). If the results are quite similar, then the SAT is highly reliable; if not, then the SAT is unreliable. To assess the level of reliability of a measurement using test-retest reliability, the researcher must use the same measurement on the same subjects. If the researcher gets the same results each time, then the measurement is highly reliable and a quality measure. If the researcher gets different results each time, then the measurement has low reliability.

Since it is common for surveys to be used in cross-sectional designs (see Chapter 5), and thus, the sample only surveyed once, test-retest reliability is used infrequently to assess the reliability of a survey. However, the process is fairly easy to complete because all the researcher needs to do is give the same survey to the same subjects within a short period of time. For example, if a researcher wanted to determine the reliability of her fear of crime survey on your campus, she could administer the survey to a classroom of students on Tuesday and the same students on Thursday. If the results for each respondent are the same, then the fear of crime survey has high reliability. If the results are substantially different, then the fear of crime survey has low reliability unless something substantial has occurred on campus (e.g., a robbery and sexual assault) or in the news that can explain the substantial differences over a two-day period.

Interrater Reliability—Reliability over Raters A measurement is reliable if someone else using the same measurement in the same situation produces almost identical results. Furthermore, a measurement is reliable if the same results are obtained time after time, even when the measurement is used by different people. Consider a blood pressure cuff. If five nurses take your blood pressure within the span of a few minutes using the same blood pressure cuff and each nurse gets the same results, then the blood pressure cuff is reliable (i.e., consistent results across different raters). The raters in this example are the nurses. The five nurses arrived at the same result, so the measurement tool (i.e., blood pressure cuff) is reliable.

Interrater reliability is particularly challenging in field research (see Chapter 6) and content analysis (see Chapter 7) where there has to be substantial agreement among those who are reading and coding the documents in content analysis or observing the behavior in field research. As it applies to surveys, interrater reliability is assessed in telephone and face-to-face interviews where the interaction between the respondent and the interviewer (i.e., rater) can influence responses. In addition, when open-ended questions are used, regardless of how the survey is administered, it becomes important that the data collected are interpreted in consistent ways. If the individuals coding the data consistently agree, then interrater reliability is high.

Split-Half Reliability—Reliability over Items A popular way of determining the reliability of a survey measurement is to assess the internal consistency of the measure. This assessment is done on indexes (i.e., when a set of survey items are developed to measure some underlying concept) by comparing answers by a respondent. A measurement is reliable over items when the items contained in the index consistently measure the underlying concept. Consider the 11-question parental attachment index, previously discussed, used by Bjerregaard and Smith. 26 Each of the 11 questions was designed to measure parental attachment. Low scores indicated low parental attachment and high scores indicated high parental attachment. How does a researcher determine the reliability, or what is sometimes referred to as the internal consistency, of the survey items?

Utilizing the split-half reliability technique, the researcher would split the items in half. This is accomplished by either selecting the responses to the first five questions and placing them in one group and placing the next five in another, or the odd-numbered questions and responses in one group and the even-numbered ones in another, or randomly selecting two sets of five. The researcher then compares the scores of the two halves on parental attachment for each respondent. If five of the items for a respondent indicate low parental attachment, then the other five should be consistent and also show low parental attachment. If the two halves correspond with one another over and over, then the index developed to measure parental attachment is a reliable one.

A statistic called Cronbach ’ s alpha is often used to assess the internal consistency/reliability of an index. The closer Cronbach’s alpha is to 1.0, the more reliable it is. The internal consistency coefficient should be at least +.60 and preferably above +.70. In an article by Bjerregaard and Smith, the authors stated, “The reliability coefficient for this scale is 0.86” 27 so the measure of parental attachment was reliable. As a consumer of research you should look for an internal consistency coefficient if the researcher uses an index. Most likely what you will see is Cronbach’s alpha, which above +.70 demonstrates a reliable measure.

Increasing Reliability

One goal of a good measure is to increase question reliability. In this section, specific ways that researchers can improve the reliability of survey questions and responses will be discussed. The focus of this section is to make you an informed consumer of research regarding reliability, not describe all the nuances of question design so you can build your own survey.

Increasing Reliability: Ensuring Consistent Meaning for All Respondents One step toward increasing reliability is to make sure each respondent in a sample is asked the same set of questions. But increasing reliability is not as simple as making sure each survey respondent is asked the same questions. Each question should mean the same thing to all respondents so each respondent interprets what is being asked in the same way. Ensuring consistent meaning for all respondents leads to consistent responses to the survey questions, thus increasing reliability. If two respondents understand the question to mean different things, their answers may be different for that reason alone, which can negatively impact the results of the study. To the extent that there is inconsistency across respondents, error is introduced, and the measurement is less reliable.

One potential problem you need to assess is whether the survey includes words that are universally understood. When surveys are done on the general population, a researcher needs to remember that those sampled will have a wide range of educational and cultural backgrounds. The researcher needs to be careful in wording the questions so they are easily understood by all participants. Also, when conducting research on a criminal justice topic, the researcher needs to be careful to not use jargon and acronyms that are used in the criminal justice system. In addition, the survey researcher needs to be sure to adequately define and describe any terms that may be uncommonly used in the general population and those terms that are sometimes used incorrectly in the general population. For example, although some have a general understanding of what activities comprise certain types of crime, it is a good idea to define each crime included in a survey. It is common for people to use burglary and robbery interchangeably, but from a survey standpoint, the researcher needs to make sure that it is clear to the respondent what activity constitutes burglary and what activity constitutes robbery by defining each crime.

Besides being sure to define any unfamiliar words used in a survey, the researcher needs to be cautious when using terms or concepts that can have multiple meanings. For example, the term “recidivism” holds multiple meanings in criminal justice research. For example, recidivism is the continuation of criminal activity, but it is measured various ways in criminal justice research. In a self-report survey, recidivism can mean any subsequent criminal activity regardless of whether the offense was reported to the police or the perpetrator was arrested. Also, if a sex offender commits a theft, is he a recidivist? The answer to this question depends on how you define recidivism. Some researchers require the subsequent offense to be as serious or more serious than the original offense to be included as recidivism. Some researchers only consider reincarceration as recidivism, while others expand it to include reconvictions, and some expand it even further by including all subsequent arrests as recidivism. Since there is so much variation in how recidivism is defined and measured, a survey researcher needs to make sure the definition of recidivism is clear in the survey so all respondents will interpret the term in the same way.

What is the reliability issue with the following survey question: Do you favor or oppose gun control legislation? The problem is that respondents will interpret the term “gun control legislation” differently, so the question will not mean the same thing to all respondents. The term “gun control legislation” encompasses many legislative strategies to control guns and there is no way of knowing which strategy the respondents are thinking about while answering the question. Gun control legislation can mean banning the legal sale of certain kinds of guns such as automatic weapons, requiring people to register their guns, limiting the number of guns that people may own, limiting which people may possess guns, or implementing a waiting period before the purchase of a gun, among others. The problem is when a respondent says “yes” they favor gun control legislation, the researcher has no way of knowing which strategy is supported by the respondent. The responses cannot be interpreted without assuming what respondents think the question means. Respondents will undoubtedly interpret this question differently. As written, the question is unreliable because it does not mean the same thing to all respondents. The solution to this problem is fairly simple. The researcher should ask a separate survey question for each gun control strategy she is interested in studying. These separate specific questions will be reliable because they will be consistently understood by all respondents and interpreted by researchers. For example, one of the survey questions can be “Do you favor legislation that limits the number of guns a person may own?” This is a reliable question that will be consistently understood by all respondents.

Increasing Reliability: Additional Question Characteristics In addition to ensuring that the question means the same thing to every respondent, there are three additional characteristics of a reliable question that you should be familiar with as you assess research articles. First, in order to be a reliable question, the researcher’s side of the question and answer process is entirely scripted so that the questions, as written, fully prepare a respondent to answer questions. In face-to-face and telephone interviews, it is important to give interviewers a script so that they can read the questions exactly as worded. Second, the kinds of answers that constitute an appropriate response to the question are communicated consistently to all respondents. The respondents should have the same perception of what constitutes an adequate answer to the question The simplest way to give respondents the same perceptions of what constitutes an adequate answer is to provide them with a list of acceptable answers by using closed-ended questions. With closed-ended questions, the respondent has to choose one of a set of appropriate responses provided by the researcher. Third, a survey question should ask only one question, not two or more. For example, a researcher should not ask the question, “Do you want to graduate from college and become a police officer?” A respondent could want to do one but not the other. When asked two questions at once, respondents will have to decide which to answer, and the survey researcher will have to assume what the respondent’s answer means.

Survey Distribution Methods

A survey researcher must decide how to distribute the survey to the sample. Until the 1970s, most academic and government surveys were completed through face-to-face interviews. 28 When telephone ownership became nearly universal in the United States, telephone interviewing became a common mechanism to distribute surveys. 29 The newest means of survey distribution is through the Internet. There are five main mechanisms to distribute a survey: mail, group-administered, Internet, face-to-face, and telephone. Each survey distribution method will be discussed in this section. The strengths and weaknesses of each distribution method will also be discussed. Survey researchers need to consider the strengths and weaknesses of each method of distribution as they decide which distribution methods works best for their research questions and survey.

Self-Administered Surveys

Self-administered surveys involve the distribution of surveys for respondents to complete on their own. Self-administered surveys include those distributed by mail, group-administered surveys, and surveys distributed via the Internet. Self-administered surveys involve minimal (i.e., group-administered) or no personal contact (i.e., mail and Internet surveys) between the researcher and the respondent. With self-administered surveys, surveys can be mailed to the sample, distributed to a large group of people in one location, or can be sent through e-mail or placed on the Internet. One of the main advantages of self-administered surveys is that the collection of sensitive, stigmatizing, and embarrassing information is more likely to be valid, as previously discussed, since respondents do not have to share their responses directly with an interviewer. However, one of the main disadvantages is that there may be a discrepancy between what people report they do and what they actually do, as previously discussed when the topic of social desirability was addressed. A survey researcher must weigh the strengths and weaknesses of each type of self-administered survey when determining the best method of distribution for their survey.

Mailed Surveys Since only a valid mailing address is required, a common method to distribute surveys is to mail the survey to the sample.

The major strengths of mail surveys are listed below.

1. Mail surveys are fairly inexpensive to complete. Since there are no interviewers to pay, the costs of a mail survey are typically much lower than a face-to-face or a telephone interview. The main cost is the printing and mailing of the surveys and the return postage for the survey responses. In addition, mail surveys can be accomplished with minimal staff and office space.

2. Mail surveys can be easily distributed to a geographically dispersed sample.

3. Mail surveys provide respondents ample time to give thoughtful answers and even to refer to other documents that may be needed to answer the survey questions.

The major weaknesses of mail surveys are listed below.

1. Mail surveys typically have lower response rates than other methods of survey distribution.

2. Mail surveys require complete and easy-to-understand instructions since there is no personal interaction between the survey researcher and the respondent. If the instructions are unclear or the survey questions are confusing to the sample, the researcher will face significant reliability issues, which will impact the results of the study as previously discussed.

3. With mail surveys, the researcher has no control over who actually completes the survey. The survey researcher may restrict the survey to only adults between the ages of 18 and 35, but that does not necessarily reflect who actually answered the survey.

Survey Shows More Teens Using Synthetic Drugs 30

Nearly one in nine high school seniors have gotten high in the past year on synthetic drugs, such as K2 or Spice, second only to the number of teens who have used marijuana, according to the Monitoring the Future survey. The survey, conducted annually by the University of Michigan, questions 47,000 students in the 8th, 10th, and 12th grades in 400 public and private schools throughout the United States. The survey is group-administered to students in their normal classrooms. It is sponsored by the National Institute of Drug Abuse and the National Institutes of Health.

Monitoring the Future, the nation’s most comprehensive survey of teenage drug use, found 11.4% of the high school seniors have used the synthetic substances, often packaged as potpourri or herbal incense and sold in convenience stores, which mimic the effects of marijuana. K2 and Spice emerged as a problem in 2008, and their popularity appears to be rising. People who smoke the chemical-coated herbs may experience euphoria, but bad reactions are common, including convulsions, anxiety attacks, dangerously elevated heart rates, vomiting, and suicidal thoughts. Most teens who smoke Spice or K2 report using other illicit drugs.

Marijuana remains the most popular drug among teens. Marijuana use increased for the fourth year in a row after a decade of decline. Nearly 7% of high school seniors report smoking marijuana daily, which is the highest rate seen in 30 years. In addition, half of high school seniors reported having tried an illicit drug at some time, 40% reported using one or more drugs in the past year, and a quarter said they had used one or more drugs in the past month, the survey found. Tobacco and alcohol use are at their lowest levels since the survey began in 1975.

Group-Administered Surveys Surveys can be distributed to individuals who have gathered together in a group. In fact, you have probably participated in a group-administered survey in one of your college classes. It is common for surveys to be administered to students in college and high school classrooms as well as to professionals in the criminal justice system who have gathered for in-service training or conference participation. The researcher will address the group regarding the purpose of the survey and to request participation. The researcher will then hand out the survey to each member of the group. The survey participants will typically complete the survey at the time of distribution and immediately return it to the researcher.

The major strengths of group-administered surveys are listed below.

1. Response rates are typically very high.

2. Since the researcher is at least present, he has the opportunity to explain the study and answer questions respondents may have about the survey.

3. Generally, group-administered surveys are inexpensive to complete since numerous surveys can be completed and returned at one time.

The major weakness of group-administered surveys is that it only applies to the small number of participants than can be easily gathered as a group. It is practically impossible to have a random sample of community residents gather to take a survey. However, some groups of people naturally congregate (e.g., students and prisoners), making group-administered surveys a viable means of survey distribution for some populations.

Internet and E-mail Surveys A recent development in survey distribution is to send an e-mail invitation to the sample. The selected subjects are then asked to answer questions by replying to a survey that is included in the e-mail or they are provided a link to a website where the survey can be completed online. Several commercial companies (e.g., Survey Monkey) make the process of creating, distributing, and collecting and coding data from an Internet survey easy and affordable. Although Internet and e-mail surveys are fairly new, the process and challenges mirror those for mail surveys.

The major strengths of Internet and e-mail surveys are listed below.

1. Internet and e-mail surveys are inexpensive to complete. The researcher does not even incur the printing, postage, and data entry costs associated with a mail survey.

2. The data from Internet and e-mail surveys can be collected quickly. An e-mail survey can be distributed instantaneously to hundreds of email addresses and since the survey is answered online, there is no delay in the researcher obtaining the completed survey. Once the respondent hits “submit” or “send” the researcher obtains the completed survey immediately.

The major weaknesses of Internet and e-mail surveys are listed below.

1. Internet and e-mail surveys are limited to individuals with e-mail addresses and Internet access. There are variations in computer ownership and use by race, ethnicity, age, income, and education, which can significantly impact the generalizability of results.

2. Depending on the sampled group and the topic, response rates to Internet and e-mail surveys are frequently low. A significant problem with Internet and e-mail surveys is inducing people to respond to the survey since there is no personal interaction between the researcher and potential respondent.

Personal Interviews

Surveys are often conducted by interviewers who read the survey items to respondents and record their responses in a face-to-face or telephone interview. A persistent issue with interviews involves the reliability (i.e., consistency) of the responses. Since there is a personal interaction between the interviewer and the respondent, the interviewer may influence the responses provided to the questions, which impacts the interrater reliability of the study. Different interviewers obtain somewhat different answers to the same questions. The interviewer’s style and personal characteristics (such as gender, race, and age) can impact the answers provided by respondents. Also, since the survey questions are read to the respondent, the interviewer’s tone and body language can lead to a different meaning to the question for different respondents. As discussed in the reliability section of this chapter, it is critical to the reliability of survey questions and responses that the questions mean the same thing to all respondents. The way certain questions are phrased by different interviewers can impact the responses given, thus impacting the reliability of the study. It is critical that interviewers are trained prior to data collection and monitored throughout the interview process so inconsistencies between interviewers can be minimized.

Face-to-Face Interviews Face-to-face interviews are typically completed in the respondent’s residence. Recall from the opening case study that the initial interview of the National Crime Victimization Survey is completed face-to-face.

The major strengths of face-to-face interviews are listed below.

1. Surveys which utilize face-to-face interviews typically have high response rates.

2. Lengthy surveys can be completed through face-to-face interviews. Although it is not always the case, some face-to-face interviews take more than one hour to complete.

3. The interviewer can answer any questions the respondent may have and can probe for additional information when an inadequate or incomplete answer is provided.

The major weaknesses of face-to-face interviews are listed below.

1. Due to the cost of interview staff and perhaps travel to the interview locations, face-to-face interviews are typically the most expensive survey distribution method.

2. The amount of time it takes to complete all the face-to-face interviews means that the data collection period is usually longer than other methods of survey distribution.

Telephone Interviews Telephone interviews are usually completed through the use of a computer-assisted telephone interviewing (CATI) system. With CATI, the interviewer reads the question from a computer screen and then enters the respondent’s answer directly into the CATI software so it is readily available for analysis. The CATI system then automatically shows the next ques tion on the computer screen. This process increases the speed with which interviews are completed and reduces interviewers’ errors. Telephone interviews are less expensive, less time consuming, and less subject to the reliability threats that can occur when conducting face-to-face interviews.

The major strengths of telephone interviews are listed below.

1. The telephone interviewing process can be completed very quickly. A staff of 15 interviewers can complete over 1,000 telephone interviews in a few days.

2. Through the use of random digit dialing (RDD), probability sampling can be utilized in telephone interviews. RDD is used because a large percentage of household telephone numbers are not listed in the current telephone book, either because they are new or at the request of the customer. RDD offers each telephone number an equal probability of being selected, which is required in probability sampling, by randomly generating the last four digits of each telephone number selected in the sample.

The major weaknesses of telephone interviews are listed below.

1. Even though RDD is used, there are sampling limitations with telephone surveys because many people do not have landline telephones. According to the 2011 National Health Interview Survey, 36.8% of the respondents did not have a working landline telephone in their home, up from 20% in 2008. 31 Furthermore, 31% of the respondents stated they receive all or almost all of their calls on cell phones. 32 People without landline phones and those who only use cell phones are generally excluded from RDD sampling. This limits the generalizability of findings generated from RDD samples.

2. In addition to the rise of cell phone use, telephone survey response rates suffer from the increased use of caller ID, which has decreased the rate at which people answer their landline telephones. Also, because many telephone calls are for sales and fundraising purposes, some people react negatively to calls from strangers, which further reduces the response rates for telephone surveys. 33

Which Method of Survey Distribution Should a Researcher Use?

It would be much easier if we just said that there is one preferred method of survey distribution, but the reality is that a survey researcher needs to consider his research questions, survey structure, population under study, and the strengths and weaknesses of each method of distribution in deciding which is best for his research project. For example, when the researcher has a limited budget, one of the self-administered techniques (i.e., mail, group-administered, Internet, and e-mail surveys) is preferable. Numerous Ph.D. dissertations in criminal justice and criminology have utilized self-administered surveys because of their low cost. However, self-administered surveys are not a good option if the researcher is concerned about the reading and writing capabilities of the sample members. Furthermore, self-administered surveys are not a good option when the survey consists of several open-ended questions. Personal interviews are more appropriate for several open-ended questions while self-administered surveys should include almost exclusively closed-ended questions. There is no simple answer to the stated question at the start of this section. The researcher needs to consider the research question and survey and then needs to maximize the strengths and minimize the weaknesses of the chosen survey distribution method.

Chapter Summary

This chapter began with a brief overview of the three main components of survey research (i.e., sampling, question design, and methods of survey distribution) followed by a discussion of nonresponse and its impact on survey results. Comprehensive discussions of validity and reliability followed. The four ways of assessing the validity of survey questions and responses were illustrated, including face validity, content validity, criterion-related validity, and construct validity. Preferred mechanisms to increase the validity of embarrassing and stigmatizing questions as well as to control for social desirability bias were explained. For reliability, the three main mechanisms to assess reliability (i.e., test-retest reliability, interrater reliability, and split-half reliability) were presented as well as mechanisms to increase reliability such as ensuring consistent question meaning for all respondents. Finally, the methods of survey distribution were presented along with the strengths and weaknesses of each.

Critical Thinking Questions

1. What are the three categories of nonrespondents, and how does nonresponse impact survey results?

2. What are the main ways to assess validity, and what strategies exist to increase the validity of survey questions and responses?

3. What are the main ways to assess reliability, and what strategies exist to increase the reliability of survey questions and responses?

4. What are the strengths and weaknesses of each survey distribution method?

construct validity: Assesses the extent to which a particular measure relates to other measures consistent with theoretically derived hypotheses concerning the concepts/variables that are being measured

content validity: When the survey questions measure the full breadth and depth of the concept being studied

criterion-related validity: An assessment to determine the strength of the relationship between the responses to the survey and another measurement, the criterion, to which it should be related if the measurement is valid

Cronbach ’ s alpha: A statistic used to assess the internal consistency/reliability of an index

face validity: An assessment of the survey questions to see if on “face value” the questions seem to be measuring the concepts and variables they are supposed to be measuring

index: A set of items that measure some underlying concept

interrater reliability: A ratio established to determine the agreement among multiple raters

reliability: Addresses the consistency of a measurement and refers to whether or not a researcher gets the same results if the same instrument is used to measure something more than once

response rate: The number of people who respond to a survey divided by the number of people sampled

scale: A set of questions that are ordered in some sequence

self-administered surveys: The distribution of surveys for respondents to complete on their own; includes surveys distributed by mail, group-administered surveys, and surveys distributed via the Internet

social desirability bias: When respondents provide answers to survey questions that do not necessarily reflect the respondent’s beliefs but instead reflect social norms

split-half reliability: An assessment of reliability in which the correspondence between two halves of a measurement is determined

telescoping: When a respondent brings behaviors and actions that occurred outside the recall period into the recall period

test-retest reliability: An assessment of reliability in which a measurement is reliable over time if the same measurement is repeated using the same subjects and yields the same results

validity: Addresses the accuracy of the measurement and refers to the extent to which researchers measure what they planned to measure

1 Truman, Jennifer L. (September 2011). Criminal victimization, 2010. Washington D.C.: Bureau of Justice Statistics.

2 Fowler, Floyd J., Jr. (2009). Survey research methods, 4th ed. Thousand Oaks, CA: Sage.

7 Truman, September 2011.

8 Fowler, 2009.

9 Earl Babbie. (2013). The practice of social research, 13th ed. Belmont, CA: Wadsworth.

10 Wilsnack, R. “Explaining collective violence in prisons: Problems and possibilities.” 61–78 in Cohen, A., G. Cole, and R. Bailey. (1976). Prison violence. Lexington, MA: Lexington Books. This study was excerpted from Amy B. Thistlethwaite and John D. Wooldredge. (2010). Forty studies that changed criminal justice: Explorations into the history of criminal justice research. Upper Saddle River, NJ: Prentice Hall.

11 Sykes, G. (1958). The society of captives: A study of a maximum security prison. Princeton, NJ: Princeton University Press.

12 Henderson, J., W. Rauch, and R. Phillips. (1987). Guidelines for developing a security program, 2nd ed. Washington D.C.: National Institute of Corrections, NIC Accession Number 006045.

13 Useem, B., and A. Piehl. (2006). “Prison buildup and disorder.” Punishment and Society 8, 87–115.

14 Fowler, 2009.

15 Rosenberg, M. (1965). Society and the adolescent self-image. Princeton, NJ: Princeton University Press.

16 ICPSR can be accessed at http://www.icpsr.umich.edu /icpsrweb/ICPSR/

17 Maltais, Michelle. (July 9, 2012). “Want a job? Check the spelling on your Facebook profile.” Los Angeles Times. http://www.latimes.com/business/technology/la-fi-social-recruiting-jobs-20120709,0,2693929.story . Retrieved on July 10, 2012.

18 Fowler, 2009.

20 LaPiere, Richard T. (1934). “Attitudes vs. actions.” Social Forces 13, 230–237.

21 Jacobs, J., and L. Kraft. (1978). “Integrating the keepers: A comparison of black and white prison guards in Illinois.” Social Problems 23, 304–318. This study was excerpted from Amy B. Thistlethwaite and John D. Wooldredge. (2010). Forty studies that changed criminal justice: Explorations into the history of criminal justice research. Upper Saddle River, NJ: Prentice Hall.

22 Jacobs and Kraft, p. 317.

23 Jurik, N. (1985). “An officer and a lady: Organizational barriers to women working as correctional officers in men’s prisons.” Social Problems 32, 375–388.

24 Bjerregaard, Beth, and Carolyn Smith. (1993). “Gender differences in gang participation, delinquency, and substance use. Journal of Quantitative Criminology 9, 329–355.

25 Hudson, W. (1982). The clinical measurement package: A field manual. Homewood, IL: Dorsey Press.

26 Bjerregaard and Smith, 1993.

27 Bjerregaard and Smith, 1993, p. 339.

28 Fowler, 2009.

30 Leger, Donna Leinwand. (December 14, 2011). “Survey: More teens using synthetic drugs.” USA Today . http://www.usatoday.com/news/nation/story/2011–12–14/more-teens-using-synthetic-drugs/51900736/1. Retrieved on June 15, 2012.

31 http://www.cdc.gov/nchs/nhis/quest_data_related_1997_ forward.htm. Retrieved on July 12, 2012.

33 Fowler, 2009.

Applied Research Methods in Criminal Justice and Criminology by University of North Texas is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License , except where otherwise noted.

Share This Book

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Sage Choice

Social Disadvantage and Crime

Per-olof h. wikström.

1 University of Cambridge, Cambridge, UK

Kyle Treiber

In this article, we analyze the relationship between social disadvantage and crime, starting from the paradox that most persistent offenders come from disadvantaged backgrounds, but most people from disadvantaged backgrounds do not become persistent offenders. We argue that despite the fact that social disadvantage has been a key criminological topic for some time, the mechanisms which link it to offending remain poorly specified. Drawing on situational action theory, we suggest social disadvantage is linked to crime because more people from disadvantaged versus affluent backgrounds develop a high crime propensity and are exposed to criminogenic contexts, and the reason for this is that processes of social and self-selection place the former more frequently in (developmental and action) contexts conducive to the development and expression of high crime propensities. This article will explore this hypothesis through a series of analyses using data from the Peterborough Adolescent and Young Adult Development Study (PADS+), a longitudinal study which uses a range of data collection methods to study the interaction between personal characteristics and social environments. It pays particular attention to the macro-to-micro processes behind the intersection of people with certain characteristics and environments with certain features – i.e., their exposure – which leads to their interaction.

‘Everybody believes that “poverty causes crime” it seems; in fact, I have heard many a senior sociologist express frustration as to why criminologists would waste time with theories outside the poverty paradigm. The reason we do… is that the facts demand it’. Robert J. Sampson (2000 : 711)

Introduction

The role of social disadvantage ( the comparative lack of social and economic resources ) in crime causation is one of the most academically and publically discussed topics in crime causation. It is difficult to imagine any criminological topic that is more debated but less scientifically understood than the extent and nature of the relationship between social disadvantage and crime (e.g., Katz, 1988 ; Sampson, 2000 , 2012 ; Tittle & Meier, 1990 ). While research findings generally suggest that social disadvantage (typically in reference to families and neighborhoods) is somehow implicated in crime causation, there is far from a simple one-to-one relationship, and researchers avidly disagree about the strength and nature of this relationship, with some even questioning whether there is a relationship at all (e.g., Agnew, 2001 , 2006 ; Bjerk, 2007 ; Braithwaite, 1979 , 1981 ; Brooks-Gunn, Duncan, & Aber, 1997 ; Coulton, Korbin, Su, & Chow, 1995 ; Dunaway, Cullen, Burton, & Evans, 2000 ; Duncan, Brooks-Gunn, & Klebanov, 1994 ; Elliott & Ageton, 1980 ; Fergusson, Swain-Campbell, & Horwood, 2004 ; Hay, Fortson, Hollist, Altheimer, & Schaible, 2007 ; Hindelang, Hirschi, & Weis, 1979 ; Hirschi, 1969 ; Janson & Wikström, 1995 ; Jarjoura, Triplett, & Brinker, 2002 ; Kornhauser, 1978 ; Loeber & Wikström, 1993 ; Messner & Rosenfeld, 1996 ; Reiss & Rhodes, 1961 ; Sampson, 1993 , 2012 ; Shaw & McKay, 1969 ; Smith, 1991 ; Tittle, 1983 ; Tittle & Meier, 1990 ; Tittle, Villemez, & Smith, 1978 ; Wikström, 1990 , 1991 ; Wikström & Butterworth, 2006 ; Wikström & Loeber, 2000 ; Wikström & Sampson, 2003 ; W. J. Wilson, 1987 ; J. Q. Wilson & Herrnstein, 1985 ; Wright et al., 1999 ).

Those who work with persistent offenders (and prisoners) on a regular basis are keenly aware that most come from disadvantaged backgrounds. In fact, any focus on offenders who hold (and warrant) the attention of the media, politicians, and practitioners (those who are more serious and/or frequent offenders) tends to validate the assumption that social disadvantage, as a common precursor, is a key cause of crime. This explains why many practitioners, policy makers, members of the general public, and even some academics perceive the relationship between social disadvantage and crime involvement to be strong and well-established. However, in focusing solely on offenders, an irreconcilable truth gets overlooked: Although most persistent offenders come from disadvantaged backgrounds, most people from disadvantaged backgrounds do not become persistent offenders . This fact may help explain why many people are convinced that social disadvantage is a main driver of crime, while research at best shows only a rather weak general (statistical) association between key indicators of social disadvantage and crime.

The main “criminological puzzle” (the key research question) is thus not why there is such a relatively weak (statistical) relationship between social disadvantage and crime (this is fairly well established) but why most persistent offenders come from a disadvantaged background, while most people from such backgrounds do not develop into persistent offenders. To answer this question requires a better understanding of the mechanisms through which social disadvantage is implicated in the development of persistent offending.

In this article, we propose to make some initial efforts to advance knowledge about the relationship between social disadvantage and crime involvement through the application of situational action theory (SAT) and the analysis of data from a random sample of U.K. adolescents from the longitudinal Peterborough Adolescent and Young Adult Development Study (PADS+).

SAT’s proposed explanation of the relationship between disadvantage and crime can be stated in three main hypotheses (see the theory section below):

• Hypothesis 1: Differences between people in crime involvement are due to differences in their crime propensity and criminogenic exposure. SAT asserts that a person’s crime propensity is essentially a consequence of his or her personal morals and ability to exercise self-control, while a setting’s criminogeneity is a consequence of its moral context (its moral norms and their enforcement).
• Hypothesis 2: Differences in crime involvement by disadvantage group are due to the fact that more people who grow up and live in disadvantaged circumstances develop a high crime propensity and are more frequently exposed to criminogenic settings.
• Hypothesis 3: Differences in the number of crime prone people and the extent of their criminogenic exposure by disadvantage group are a consequence of disadvantage-related differences resulting from (rules and resource based) social and self-selection processes.

The first (situational) hypothesis has been thoroughly studied and supported in our previous research (see Wikström, Oberwittler, Treiber, & Hardie, 2012 ). In this article we will focus mainly on empirically exploring the second hypothesis that disadvantage-related differences in crime involvement are primarily due to disadvantage-related differences in the number of crime-prone people and their level of exposure to criminogenic settings. In other words, we assume that when controlling for crime propensity and criminogenic exposure, any predictive effect of disadvantage will vanish. If we are correct in this assumption, we assume that this is due to the fact that more people from disadvantaged backgrounds have been exposed to developmental contexts that promote the development of a stronger crime propensity and that more people in disadvantaged circumstances are exposed to criminogenic settings (Hypothesis 3). In this study, we can partially test the latter hypothesis by exploring if people from disadvantaged circumstances spend more time in criminogenic settings. However, we will not explore whether those with higher crime propensity have a history of exposure to developmental settings promoting a higher crime propensity. This assumption remains to be tested in future studies.

Situational Action Theory

SAT defines crime as acts that break rules of conduct stated in law and analyzes crime as moral actions; that is, as actions guided by rules about what actions are right or wrong under particular circumstances, with the law being seen as one among many sets of rules of conduct that guide people’s actions (e.g., Wikström, 2006 , 2010 ). The framework of SAT is briefly summarized in the following key propositions and illustrated in Figure 1 .

The theoretical framework of situational action theory.

Source: Wikström, P-O. H. (2011) . Does everything matter? Addressing the problem of causation and explanation in the study of crime. In J. McGloin, C. J. Sullivan, & L. W. Kennedy (Eds.), When crime appears: The role of emergence (pp. 53-73). London, England: Routledge.

• Proposition 1: Crime is ultimately an outcome of a perception–choice process .
• Proposition 2: This perception–choice process is initiated and guided by relevant aspects of the person–environment interaction.
• Proposition 3: Processes of social and self-selection place kinds of people (those with certain personal characteristics) in kinds of settings (those with certain environmental and circumstantial features), creating particular kinds of interactions.
• Proposition 4: What kinds of people and what kinds of settings are present in a jurisdiction is the result of historical processes of personal and social emergence.

Propositions 1 and 2 refer to the situational model, and Propositions 3 and 4 to the social model, of SAT. Figure 1 illustrates how these two models are linked. SAT proposes that the causes of action (such as acts of crime) are situational (Propositions 1 and 2) and that the social factors affecting people’s crime involvement (i.e., factors influencing processes of emergence and selection) are best analyzed as causes of the causes (Propositions 3 and 4). In essence, SAT argues that people commit crime because they come to see and choose (habitually or deliberately) an act of crime as an action alternative. The key situational factors and processes involved are illustrated in Figure 2 . The basic idea is that motivation (temptations and provocations) initiates the action process (being a necessary but not sufficient factor), the moral filter (which depends on the interaction between personal morality and the moral norms of the setting) endorses action alternatives in response to a particular motivation, and controls (self-control and deterrence) influence the process of choice but only when the actor deliberates because there is conflicting rule guidance regarding crime as an action alternative (for details of the role of these factors and processes, see, e.g., Wikström, 2011 ; Wikström et al., 2012 ).

Key situational factors and processes in crime causation according to situational action theory.

Source : Wikström, P-O. H. (2011) . Does everything matter? Addressing the problem of causation and explanation in the study of crime. In J. McGloin, C. J. Sullivan, & L. W. Kennedy (Eds.), When crime appears: The role of emergence (pp. 53-73). London, England: Routledge.

SAT argues that to understand how social factors (like social disadvantage) and developmental factors (such as cumulative experiences of disadvantage-related social conditions) influence people’s crime involvement as “causes of the causes” (i.e., causes of why people develop a high crime propensity and why settings develop weak law-relevant moral contexts), we need to understand how historical processes of social and personal emergence and contemporaneous processes of social and self-selection come to influence how people ultimately see their action alternatives and make their choices by exposing them to particular settings (environments) in which they develop and act (see also Wikström et al., 2012 ).

We posit that social disadvantage affects people’s crime involvement primarily through (rule- and resource-based) processes of social and self-selection which influence their exposure to crime-relevant developmental and action settings. We define selection as social and personal forces (dependent on social and personal resources and rules) that enable (encourage or compel) or restrict (discourage or bar) particular kinds of people from taking part in particular kinds of developmentally and action-relevant time and place-based activities (for more details, see Wikström et al., 2012 ). Specifically, we propose social and self-selection processes lead to young people from disadvantaged backgrounds being more profoundly exposed to (a) settings and circumstances which lead them to develop and sustain a high crime propensity (weak personal morality and a lack of ability to exercise self-control) and (b) moral contexts that are conducive to engagement in acts of crime (i.e., those in which rules of law are loosely applied and/or weakly enforced).

Thus, our proposed answer to the question, “What is the relationship between social disadvantage and crime?” is the following: The impact of social disadvantage on young people’s crime may be primarily through disadvantage-induced selection processes which place disadvantaged young people more often than others in developmental contexts that are conducive to the development of a higher crime propensity, and in action contexts in which acts of crime tend to be encouraged (or at least are not strongly discouraged) .

The Peterborough Adolescent and Young Adult Development Study (PADS+)

PADS+ is a longitudinal study that has followed a random sample of 716 young people who were living in the city of Peterborough in 2002, since they were 12 years old (2003), through adolescence, and now into young adulthood ( Figure 3 ). Data used in this article were collected annually from the young people between 2004 and 2008, although additional waves have been completed at ages 19 (2010), 21 (2012), and 24 (2015). Methods included an extensive interviewer-led questionnaire, cognitive measures, a life events calendar, randomized scenarios, and a space–time budget. Data are also taken from an initial wave of data collected from participants’ parents in 2003 via a structured interview, including in-depth information about participants’ families’ social situations at the time of their enrolment in the study and retrospective information on their childhood experiences and critical life events. Over the period analyzed in this article, an exceptionally high retention rate was maintained, with 97% of the sample taking part in all five waves (ages 13-17). For this study, we focus on the 657 young people (92% of the sample) who took part in all five waves and completed space–time budgets in all five waves (for those who moved outside the study area, only time-budget data are available, which hinders some analyses). These young people were similar on all key variables, including crime involvement, except that those who were lost had significantly higher neighborhood disadvantage at age 12 (mean = 1.0 vs. 0.51, p = .001).

Overview of the PADS+ (Peterborough Adolescent and Young Adult Development Study) research design.

In addition to data collected from the main cohort study, two special small-area community surveys were carried out, one in 2005 and another in 2012, each with independent samples of around 6,000 randomly selected Peterborough residents aged 18 years or older, to gather data on social environments (e.g., levels of social cohesion and informal social control). This article draws on the 2005 survey, as well as external data from the 2001 U.K. Census. Data not analyzed in this article have also been collected from key social agencies (e.g., the police, probations service, schools, etc.). Figure 4 gives an overview of the PADS+ design and key methodologies; for a detailed presentation, including sampling methods and descriptives, data sources, and data quality, see Wikström et al. (2012) .

Overview of key PADS+ (Peterborough Adolescent and Young Adult Development Study) methodologies.

In this article, we focus on the adolescent time window, which is of particular interest criminologically due to its encompassing most people’s entry into, escalation during and beginning of desistence out of, crime involvement. Adolescence is also particularly interesting in regard to changes in exposure to different social environments as young people shift their focus from family to peer activities and begin to establish their own autonomy through their social lives outside the home. Our hypothesis that social disadvantage influences young people’s crime involvement through its effects on the kinds of settings they are exposed to is highly relevant in the adolescent context, as it relates to differences in access to settings (e.g., what kind of settings are more proximate and young people’s mobility) as well as the kinds of settings they may choose to take part in (how they spend their time) as their autonomy increases. With five years of data on adolescents, including their experiences of social disadvantage, their changing activity fields and their patterns of crime involvement, PADS+ provides an excellent opportunity to study the selective influence of social disadvantage and its repercussions for young people’s social lives during adolescence, and on into young adulthood.

Creating a Baseline Measure of Childhood Social Disadvantage

Most studies of disadvantage focus on either family or neighborhood disadvantage. However, it has been noted that both capture important and distinct aspects of a person’s lack of economic and particularly social resources (e.g., Hay et al., 2007 ). We have accordingly developed a combined measure of family and neighborhood disadvantage using robust measures of each to comprehensively assess personal differences in early disadvantage.

Family Disadvantage

A family disadvantage index was constructed from three measures covered in the parents’ interviews, reflecting the family’s situation when the young people were 12 years old. It has been highlighted in previous research that young people are often not reliable informants of their family’s disadvantage ( Duncan et al., 1994 ); by asking their parents, we acquire a much more accurate measure. We have included three key indicators in our family disadvantage measure: participants’ family’s household income, 1 their parents’ highest educational level, 2 and their parents’ highest occupational class 3 . Principal component analysis was employed to ensure these three variables represent one latent factor and factor scores were assigned to each participant.

Neighborhood Disadvantage

A neighborhood disadvantage index was constructed from 2001 U.K. Census data for the output area of the young person’s main home at the time of the parents’ interviews (2003), when participants were 12 years old. This measure is comparable with the Indices of Multiple Deprivation 2004 which measures disadvantage at a larger area level. This index included four items: the percentage of area residents who were working class, 4 the percentage of area residents with no or low educational qualifications, 5 the percentage of residents who were unemployed; and the percentage of area residents who resided in detached houses. Principal component analysis was employed to ensure these four variables represent one latent factor; as expected, all four loaded on a single factor and all loadings were greater than .50 (see Wikström et al., 2012 , for details). Factor scores were assigned to each output area, and subsequently, each participant was assigned the factor score of their home output area.

Combined Disadvantage

Factor scores for family and neighborhood disadvantage were standardized (family scores across participants and neighborhood scores across output areas) and summed to create a combined disadvantage score for each participant, representing his or her experience of family and neighborhood disadvantage at age 12. For many analyses, combined disadvantage is divided into five equal-sized groups (20 percentiles). Family and neighborhood disadvantage were moderately correlated ( Figure 5 ; r = .55). While the most disadvantaged families lived in areas with varying levels of neighborhood disadvantage, few of the most advantaged families lived in highly disadvantaged neighborhoods.

Scatter plot of family and neighborhood disadvantage.

The combined family and neighborhood disadvantage scale has a reasonably approximately normal distribution and is consistent with other indicators of disadvantage measured in the parents’ interviews and an additional site survey conducted just after by PADS+ researchers. For example, families scoring highly on the combined disadvantage index were more likely at age 12 to live in a house whose interior, exterior, and surrounding area were in poor or very poor condition; to not be living in their family of origin; to have more biological and nonbiological siblings; to be an unplanned child; and to have a mother who suffered from postnatal depression.

Measuring Social Environments and Activity Fields

One of the biggest shortcomings in criminology is a lack of adequate research into the role of social environments, driven in part by a lack of adequate data. There has been surprisingly little significant advancement in methods used to study environments in criminology since the works of the early Chicago School (e.g., Shaw & McKay, 1969 ; see Sampson, Morenoff, & Gannon-Rowley, 2002 ). The main exceptions are the recent introduction of large-scale community surveys (e.g., Sampson, Raudenbush, & Earls, 1997 ; Sampson & Wikström, 2008 ; Wikström, Torstensson, & Dolmen, 1997 ) and the development of ecometrics, a method for assessing the reliability of measures of environments, such as neighborhoods ( Raudenbush & Sampson, 1999 ).

When social environments have been analyzed longitudinally, the focus has traditionally remained on the family and (to a lesser degree) school environments, ignoring neighborhoods (see, e.g., Brooks-Gunn, Duncan, Klebanov, & Sealand, 1993 ). The few recent longitudinal studies which have explored neighborhood effects (e.g., the Project on Human Development in Chicago Neighborhoods, or PHDCN) still overlook the role of people’s exposure to the wider environment. They equate a person’s environment only with his or her own neighborhood. However, findings from PADS+ show that people spend a lot of their time outside their neighborhoods and that people living in the same neighborhood may be exposed to very different kinds of environments (see, e.g., Wikström, Ceccato, Hardie, & Treiber, 2010 ). One reason studies fail to capture this variation in exposure is because they traditionally use a geographical unit of analysis which is too large and heterogeneous to adequately reflect the part of the environment that influences people’s actions and development (i.e., the part they experience with their senses); for example, most studies define neighborhoods using areas containing thousands of residents.

PADS+ was specifically designed to help overcome these limitations using new methodologies in combination with more established methods of measuring personal characteristics and experiences. These methods aim to

• Measure the part of the environment which people directly experience ( using small-area units of analysis )
• Reliably measure relevant aspects of the social environment ( using ecometrics ; see Raudenbush & Sampson, 1999 )
• Account for the fact that people move around in space and encounter a wide range of different environments outside their neighborhoods ( using space–time budget methods )

To measure young people’s exposure to different environments, PADS+ uses a strategy that combines (geographically matches) data from a small-area community survey (and official data on population composition and land use at the same small-area level) with data from a space–time budget ( Wikström, Treiber, & Hardie, 2011 ). The community survey collects data from residents across the study area concerning social environmental variables such as social control and cohesion. These data are then linked geographically to data from the Census and other official databases to characterize areas using the smallest available unit, an output area (with, on average, 124 households), which, in turn, is linked to data from the space–time budget.

A space–time budget gathers very detailed time-diary data linked to a spatial unit and can therefore be used to calculate complex measures of exposure to (time spent in) a range of settings. The method includes hundreds of detailed codes for geographical locations, functional places (e.g., street corner), activities (e.g., skateboarding), and who a person is engaging with (e.g., peers) which combine to characterize a setting, as well as codes for additional circumstances including involvement in crime (as victim or offender) and substance use (see Wikström et al., 2012 , pp. 70-75 and Technical Appendix A2, for details of these data). For each participant, PADS+ collects detailed space–time budget data about each hour over four days each wave (the Friday, Saturday, and two weekdays preceding the interview). Each wave comprises on average more than 65,000 hours of space–time budget data (amounting to more than 300,000 hours of data for the period analyzed in this article; 480 hours of data for each participant).

Data collected through these methodologies (space–time budgets combined with small-area community surveys) then allows the exploration of patterns of interaction between people’s exposure to different environments and their personal characteristics (e.g., using data from the interviewer-led questionnaires).

Criminogenic Exposure

Criminogenic exposure is a composite index of two scales: exposure to criminogenic settings and peers’ crime involvement .

Exposure to criminogenic settings is measured using space–time budget and social environmental data and refers to how many hours a person spent in unstructured peer-oriented activities in local and city centers or areas with poor collective efficacy .

Peer-oriented activities are defined as activities that take place outside of school and work settings in the presence of peers with no adult guardians present. Peers are frequently linked to crime involvement and may significantly influence crime involvement via their impact on the moral context (e.g., inducements to offend, relevant rules, and levels of enforcement). Of course peers can also strengthen the moral context, and this is taken into account by qualifying exposure to criminogenic settings according to levels of peers’ crime involvement (see below). Lack of supervision is a well-known predictor of crime involvement and can weaken the moral context particularly by reducing levels of enforcement.

Unstructured peer-oriented activities are peer-oriented activities which lack any goal-direction and mainly involve media consumption and socializing (see Wikström et al., 2012 for more details on this and other time use variables).

Collective efficacy refers to the capacity and willingness of people (typically residents) of a given area to act communally, for example, to hold and uphold a shared set of moral norms, and is supported by social cohesion and informal social control ( Sampson et al., 1997 ; Sampson, Morenoff, & Earls, 1999 ). Areas with strong collective efficacy maintain and effectively enforce shared norms, while areas with poor collective efficacy are characterized by inconsistent norms and an inability to informally monitor and shape the conduct of area users. Collective efficacy is measured using two indices from the 2005 community survey in which residents living in each output area were asked about local social cohesion (five items tapping how much people in the neighborhood get along and share similar values) and informal social control (four items tapping the likelihood that neighbors would intervene if young people were breaking rules). Responses to these items were summed and averaged across respondents for each output area, then standardized and summated. See Wikström et al. (2012) for a description of reliability analyses used to validate this scale.

Commercial and entertainment centers , such as city and local centers, experience heavy traffic by nonresidents who move continuously in and out. The variety of setting users contributes to a lack of shared norms, and the sheer number of users reduces the ability for those users to monitor and control behavior; hence, the collective efficacy of area residents may not adequately reflect their moral contexts (see Wikström et al., 2012 ). Crime-relevant motivators, such as temptations which are presented by commercial venues and frictions which occur when different groups of people come into contact, especially in the presences of alcohol and other drugs, also tend to be concentrated in commercial and entertainment districts. Therefore, we separate these areas from residential areas in our analyses. City and local centers refer to a total of 10 output areas which comprise the main city center of Peterborough (four output areas) and four local centers (six output areas in total).

A composite exposure to criminogenic settings variable was created as the total number of hours recorded during the four days captured by the space–time budget in each wave which were spent in unstructured peer-oriented activities in local or city centers or in areas with poor collective efficacy.

Peers’ crime involvement was measured using a six-item scale asking if a person’s peers engaged in different acts of rule breaking (“no, never,” “yes, sometimes,” “yes, often [every month],” and “yes, very often [every week]”) including skipping school or work, getting drunk, substance use, shoplifting, vandalism, and fighting (see Wikström et al., 2012 , for further detail). For each wave, participants received a summated peer crime involvement score.

The final measure of criminogenic exposure was created by standardizing participants’ criminogenic setting and peers’ crime involvement scores for each wave across all waves (3,000+ scores for each variable). Each participant’s five standardized scores for each variable were then summed to create a composite criminogenic exposure score for the entire period (ages 13-17). This scale was positively skewed, with most participants reporting less criminogenic exposure.

In some analyses, we refer to low, medium, and high criminogenic exposure. These categories are determined by standard deviations: high and low exposure refer to scores more than one standard deviation above and below the mean, respectively.

Crime Propensity

Crime propensity represents an additive index of standardized scores on two scales: personal moral rules and generalized ability to exercise self-control .

Personal moral rules were measured by asking participants to rate whether they thought 16 acts of rule breaking were “very wrong,” “wrong,” “a little wrong,” or “not at all wrong.” These acts ranged from minor to more serious acts (e.g., teasing a classmate to breaking into a building to steal something). Details are provided in Wikström et al. (2012) . Item responses were summed to create an index for each wave.

Generalized ability to exercise self-control was measured using eight items which asked participants to rate whether they “strongly agree,” “mostly agree,” “mostly disagree,” or “strongly disagree” with statements about themselves (e.g., “I never think about what will happen to me in the future”; “I lose my temper pretty easily”). This is a variant of Grasmick, Tittle, Bursik, and Arneklev’s (1993) scale which relies on items more consistent with SAT’s conceptualization of the expression of the ability to exercise self-control. As with the personal moral rules scale, self-control item responses were summed to create an index for each wave.

A composite crime propensity measure was then created by standardizing moral rules and self-control scores for each participant in each wave across all waves (3,000+ scores for each variable). Standardized scores for each variable from each wave were then summed to create crime propensity scores for each participant for the entire period (ages 13-17). Crime propensity was reasonably normally distributed across the study period.

As with criminogenic exposure, in some analyses, we refer to low, medium, and high crime propensity. These categories are determined by standard deviations: high and low propensity refer to scores more than one standard deviation above and below the mean, respectively.

Crime involvement tapped into participants’ prevalence and frequency of engaging in nine acts of crime which are most representative of young people’s crime: arson, vandalism, theft from a person, shoplifting, assault, robbery, car crime (theft of or from a car), residential burglary, and nonresidential burglary. In all waves, participants were asked if they had committed the act during the previous calendar year and if so, how many times they had done so (for more details on these methods and general patterns of the sample’s crime involvement, see Wikström et al., 2012 ). From these items, prevalence and frequency measures were derived by wave and across all five waves (ages 12-16, as crime were reported for the previous year).

More than two thirds (70%) of the sample reported at least one act of crime during the study period, though most reported only one or two. Crime prevalences peaked at age 14, but for those who continued offending, crime frequencies increased steadily to age 16. A small proportion of the sample (4% or 27 young people) reported more than 100 acts of crime during the study period and was responsible for nearly half of all crimes reported (7,523 out of 15,970 crimes). These persistent offenders are specifically identified in some analyses. Assault was the most commonly reported act, with more than half of the sample reporting having hit or beaten up someone during the study period. For more details on the sample’s crime involvement between ages 12 and 16, see Wikström et al. (2012) .

• Are young people from disadvantaged backgrounds more heavily involved in crime?

The crucial answer is ‘not necessarily.’ As the distribution in Figure 6 shows, the vast majority of young people in the sample report few acts of crime, regardless of their preadolescent experiences of disadvantage, hence the correlation between disadvantage and crime involvement is decidedly small ( r = .20 when crime is logged, .14 when not). Clearly, even a combined index of neighbourhood and family disadvantage is not good predictor of crime involvement, accounting for only 4% of the variance.

Scatter plot of combined family and neighborhood disadvantage at age 12 and crime frequency from ages 12 to 16 (logged).

And yet PADS+ data also clearly illustrates the key criminological puzzle that was the impetus for this paper ( Figure 7 ): a substantial majority (70.4%) of the sample’s persistent offenders came from disadvantaged backgrounds, but the majority of young people from disadvantaged backgrounds (93%) did not become persistent offenders.

Venn diagram showing the overlap between young people from disadvantaged backgrounds and persistent offenders.

• If disadvantage is not the answer, what is (Hypothesis 1)?

Table 1 presents another illustration of the modest trend towards increasing crime prevalence and frequency with increasing social disadvantage by grouping participants into five equal sized (20 percentile) groups based on their disadvantage at age 12. If we compare these trends to those observed if the sample is divided instead into equal groups based on their crime propensity or criminogenic exposure, the key variables SAT proposes are directly linked to crime involvement (Hypothesis 1), we can see that the relationship between social disadvantage and crime pales by comparison.

Crime Involvement (Ages 12-16) by Participants’ Combined Disadvantage at Age 12, Crime Propensity, and Criminogenic Exposure (Divided Into Five Equal Groups).

• Do young people from disadvantaged backgrounds have higher crime propensity and more exposure to criminogenic settings (Hypothesis 2)?

We have hypothesized that differences in crime involvement by disadvantage group are due to the fact that more people who grow up and live in disadvantaged circumstances develop a high crime propensity and are more frequently exposed to criminogenic settings. Table 2 shows this is indeed the case. On average, young people from disadvantaged backgrounds have a higher crime propensity (weaker personal morality and ability to exercise self-control; r = .19) and greater criminogenic exposure (more crime prone peers and exposure to criminogenic settings; r = .27) (see Figure 8 ).

Mean Scores for Crime Propensity and Criminogenic Exposure (Ages 13-17) by Participants’ Combined Disadvantage at Age 12 (Five Equal Groups).

Scatter plot of participants’ combined family and neighborhood disadvantage at age 12 and crime propensity and criminogenic exposure between ages 13 and 17.

Although young people from disadvantaged backgrounds tend to have higher crime propensity and more criminogenic exposure on average, there is variation in both propensity and exposure at all levels of disadvantage, and the relationships between propensity and exposure and crime involvement remain strong regardless ( Table 3 and Table 4 ). Importantly, there were no consistent substantive differences in crime involvement by disadvantage group among participants with different levels of crime propensity or criminogenic exposure (i.e., the effects of disadvantage on crime involvement appear to be mediated by propensity and exposure).

Crime Propensity Groups (Ages 13-17) by Participants’ Combined Disadvantage at Age 12 (Five Equal Groups).

Criminogenic Exposure Groups (Ages 13-17) by Participants’ Combined Disadvantage at Age 12 (Five Equal Groups).

Is social disadvantage a cause of the causes of crime?

The suggestion that the relationship between social disadvantage and crime involvement is fully mediated by differences in young people’s crime propensity and exposure to criminogenic settings (Hypothesis 2) is borne out by the regression models shown in Table 5 . Disadvantage at age 12 significantly predicts crime involvement between ages 12 and 16, but explains very little of the variance in crime frequency (3.9%); by contrast, propensity and exposure together explain 55.6% of the variance, and fully mediate the effects of both family and neighbourhood disadvantage.

Disadvantage, Propensity, and Exposure Predicting Crime Involvement (Number of Crimes Logged).

• How do activity fields differ for young people from disadvantaged backgrounds (Hypothesis 3ish) ?

We have suggested that differences in the number of crime prone people and the extent of their criminogenic exposure by disadvantage group is a consequence of disadvantage-related differences in social and self-selection which expose more young people from disadvantaged backgrounds to more criminogenic developmental and action contexts (Hypothesis 3). Although we will not test these processes directly, we can preliminarily assess differences in exposure to settings and circumstances and shed light on these processes in action ( Table 6 ).

Participants’ Time Use (Ages 13-17) by Their Combined Disadvantage at Age 12 (Five Equal Groups).

One very significant difference in time use by disadvantage groups relates to the time they spend in educational activities; those from disadvantaged backgrounds spend significantly less time in educational activities and at school than those from more advantaged backgrounds (for example, those in the highest disadvantage group spent, on average, nearly 1.5 hours less per day in educational activities – e.g., attending classes, doing homework – across the study period than those in the lowest disadvantage group). Substantive differences are evident at every age, though the most dramatic differences appear once the young people have reached the age at which they may leave compulsory education.

By contrast, young people from disadvantaged backgrounds spend more time on average in leisure activities, including socialising (close to one hour more per day across the study period), than young people from more advantaged backgrounds, and more of this time is unstructured. Disadvantaged young people also spend more time on average unsupervised, and in particular unsupervised with their peers (nearly half an hour more per day), and those peers are more likely to be crime prone.

Young people from disadvantaged backgrounds also spend considerably more time on average in areas with poor collective efficacy and less in area with strong collective efficacy. This is in part driven by the area characteristics of their home neighbourhoods, but disadvantaged young people also spend twice as much time outside their home neighbourhoods in areas with poor collective efficacy (two hours on average per day). They do not, however, spend much more time in the city and local centres. Overall, socially disadvantaged young people’s exposure to settings theorized to be conducive to crime is significantly higher on average than that of those who are less disadvantaged. However, it is important to note that this still represents only a fraction (less than 5%) of their time awake.

• Do young people from disadvantaged backgrounds who do not offend have lower crime propensity and less criminogenic exposure than those who do ?

We have drawn attention to the fact that not all young people from disadvantaged backgrounds commit acts of crime. If the explanation we have posited is correct—that is, disadvantage influences crime involvement via selection processes which lead to higher crime propensity and greater exposure to criminogenic settings—we would expect that these effects are attenuated for young people who come from disadvantaged backgrounds who do not offend. This is precisely what we find ( Table 7 ). Young people from disadvantaged backgrounds who do not offend demonstrate average levels of personal morality and ability to exercise self-control more consistent with young people from the least disadvantaged backgrounds. Their time use as well is dramatically different from that of those from disadvantaged backgrounds who are crime involved, as is their peers’ crime involvement; they spend less time unsupervised with their peers and in areas with poor collective efficacy, and their peers are much less frequently involved in crime. These effects are not explained by differences in their experiences of disadvantage; they do not differ in their average family or neighborhood disadvantage or experience significant changes in their situations during the study period.

Comparison of means for offenders and non-offenders with similar experiences of disadvantage.

A complementary assumption which can be drawn is that young people in the least disadvantaged areas who do offend would likewise have a higher crime propensity and criminogenic exposure than their nonoffending peers. We find this is also the case, although the differences are perhaps not as extreme as those between offending and nonoffending young people from disadvantaged backgrounds. Interestingly, one difference which arises is that while disadvantaged young offenders and nonoffenders differed in their exposure to areas with poor collective efficacy, but not significantly to city and local centers, the opposite is true for advantaged young offenders and nonoffenders; offenders from the least disadvantaged backgrounds spent significantly more time in the city and local centers than their nonoffending peers, but no more time in areas with poor collective efficacy. These differences in time use have interesting implications for understanding the impact of differences in mobility and access to settings between advantaged and disadvantaged young people, as well as for differences in the kinds of settings and circumstances which they may find crime conducive.

• Are young people from disadvantaged backgrounds more likely to have a high propensity AND high criminogenic exposure? Do young people from all backgrounds with a higher crime propensity AND criminogenic exposure commit more crime?

The main premise of SAT is that it is the interaction between crime propensity and criminogenic exposure that is the key to understanding crime involvement. The final analysis focuses in on this interaction. Table 8 shows that, as expected, more young people from disadvantaged backgrounds demonstrate a high crime propensity and experience a high rate of exposure to criminogenic settings than those from less disadvantaged backgrounds. However, regardless of their levels of disadvantage, young people with a high crime propensity and high criminogenic exposure report high rates of crime involvement (practically 100%) and extremely high crime frequencies. Figure 9 illustrates this pattern very clearly. The relationship between crime propensity and criminogenic exposure and crime involvement are remarkably similar across disadvantage groups: regardless of levels of disadvantage, as propensity and exposure increase, crime involvement increases (and the amount of explained variance is practically identical). The only substantial difference is that at lower levels of disadvantage, fewer young people experience a high crime propensity and high criminogenic exposure.

Cross-Table of Crime Propensity and Criminogenic Exposure (Ages 13-17; Groups by SD ) Split by Combined Disadvantage (Five Equal Groups) Showing Distributions and Crime Involvement.

Scatter plots of crime propensity and criminogenic exposure (ages 13-17) by combined disadvantage (total and for five equal groups).

Conclusions

The relationship between social disadvantage (the comparative lack of social and economic resources) and crime has been a persistent puzzle for criminologists. Consistent with most previous research, we have found that coming from a disadvantaged background was not a strong predictor of crime involvement in our sample, even when using a combined measure of family and neighbourhood disadvantage, and yet a large proportion of our most persistent offenders did indeed come from disadvantaged backgrounds.

We have presented, and tested, a new explanation for this seeming contradiction. First we have suggested that differences between people in their crime involvement are due to differences in their crime propensity and criminogenic exposure, and we have shown that crime propensity and criminogenic exposure are stronger and more consistent predictors of crime involvement than social disadvantage which fully mediate the relationship between social disadvantage and young people’s crime.

Second, we have suggested that differences in crime involvement by disadvantage group are due to the fact that more people who grow up and live in disadvantaged circumstances develop a high crime propensity and are more frequently exposed to criminogenic settings. We have shown this to be the case. We have also shown that the relationship between crime propensity, criminogenic exposure and crime involvement remains remarkably consistent and robust regardless of participants’ levels of disadvantage.

Finally, we have suggested that these differences in the number of very crime prone people and the extent of their criminogenic exposure by disadvantage group are a consequence of disadvantage-related differences resulting from (rules and resource based) social and self-selection processes. We have shown that young people’s time use differs significantly in criminogenic ways depending on their levels of disadvantage supporting our selection hypothesis. In future publications we will further explore the selection processes which influence young people’s activity fields, both in relation to developmental and action contexts, with the aim of better understanding why people vary in their crime propensity and criminogenic exposure, including what environmental qualities other than social and economic resources may explain such variations.

Our overall conclusion is that social disadvantage is only moderately related to factors related to crime involvement (crime propensity and criminogenic exposure). Our findings support the assertion that the relationship between social disadvantage and crime involvement may be explained by the fact that more young people who experience childhood disadvantage at home and in their neighbourhoods are likely to develop a high crime propensity and be exposed to criminogenic settings, but that these are far from inevitable outcomes of growing up disadvantaged.

Author Biographies

Per-Olof H. Wikström (PhD, Docent, Stockholm University), FBA, is Professor of Ecological and Developmental Criminology at the University of Cambridge Institute of Criminology. He is the director of the Peterborough Adolescent and Young Adult Development Study (PADS+) and his main research interests are developing unified theory of the causes of crime (Situational Action Theory), its empirical testing (PADS+) and its application to devising knowledge-based prevention policies.

Kyle Treiber is a Lecturer in Neurocriminology at the University of Cambridge Institute of Criminology and Deputy Director of Peterborough Adolescent and Young Adult Development Study (PADS+). She has an educational background in criminology and neuropsychology. Her research interests include criminal decision making and the interaction between biological and social factors in development and action.

1. Before taxes; increments of £5,000 up to £70,000+.

2. University degree (three levels: doctorate, graduate, or undergraduate); A levels (or vocational qualifications such as a BTEC [Business and Technical Education Council] extended diploma or HND [Higher National Diploma]); O levels (compulsory education); not completed compulsory education.

3. A four-class scale divided into lower working (unskilled workers), working (skilled workers and low-ranking white-collar employees), lower middle (officials and small-scale entrepreneurs), and upper middle and upper (large-scale entrepreneurs, high-ranking officials, and high-ranking white-collar employees).

4. National Readership Survey.

5. National Qualifications Framework.

Declaration of Conflicting Interests: The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding: The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Economic and Social Research Council [grant number ES/K010646/1].

Live revision! Join us for our free exam revision livestreams Watch now →

Criminology

Reference Library

Collections

• See what's new
• All Resources
• Student Resources
• Assessment Resources
• Teaching Resources
• CPD Courses
• Livestreams

Study notes, videos, interactive activities and more!

Criminology news, insights and enrichment

Currated collections of free resources

Browse resources by topic

• All Criminology Resources

Resource Selections

Currated lists of resources

Police recorded crime statistics - strengths and limitations

Police recorded statistics measure acts that are deemed to be criminal and are not reliant upon the victim’s interpretation of whether it was a crime.

Police recorded statistics are compiled by official government agencies each year and can be compared with previous years.

Police recorded statistics can help to decide policing budgets by predicting the level of crime that the police will have to deal with in the coming year.

Weaknesses:

Police recorded statistics are subject to the changing priorities of government agencies and some crimes may not be considered worthy of investigation.

Police recorded statistics may be manipulated to show falling levels of crime to meet government targets.

Not all crimes are reported to the police, nor are all crimes reported to the police recorded as offences.

• Share on Facebook
• Share on Twitter
• Share by Email

Our subjects

• › Criminology
• › Economics
• › Geography
• › Health & Social Care
• › Psychology
• › Sociology
• › Teaching & learning resources
• › Student revision workshops
• › Online student courses
• › CPD for teachers
• › Livestreams
• › Teaching jobs

Boston House, 214 High Street, Boston Spa, West Yorkshire, LS23 6AD Tel: 01937 848885

• › Contact us
• › Terms of use
• › Privacy & cookies

© 2002-2024 Tutor2u Limited. Company Reg no: 04489574. VAT reg no 816865400.

How to Write Limitations of the Study (with examples)

This blog emphasizes the importance of recognizing and effectively writing about limitations in research. It discusses the types of limitations, their significance, and provides guidelines for writing about them, highlighting their role in advancing scholarly research.

Updated on August 24, 2023

No matter how well thought out, every research endeavor encounters challenges. There is simply no way to predict all possible variances throughout the process.

These uncharted boundaries and abrupt constraints are known as limitations in research . Identifying and acknowledging limitations is crucial for conducting rigorous studies. Limitations provide context and shed light on gaps in the prevailing inquiry and literature.

This article explores the importance of recognizing limitations and discusses how to write them effectively. By interpreting limitations in research and considering prevalent examples, we aim to reframe the perception from shameful mistakes to respectable revelations.

What are limitations in research?

In the clearest terms, research limitations are the practical or theoretical shortcomings of a study that are often outside of the researcher’s control . While these weaknesses limit the generalizability of a study’s conclusions, they also present a foundation for future research.

Sometimes limitations arise from tangible circumstances like time and funding constraints, or equipment and participant availability. Other times the rationale is more obscure and buried within the research design. Common types of limitations and their ramifications include:

• Theoretical: limits the scope, depth, or applicability of a study.
• Methodological: limits the quality, quantity, or diversity of the data.
• Empirical: limits the representativeness, validity, or reliability of the data.
• Analytical: limits the accuracy, completeness, or significance of the findings.
• Ethical: limits the access, consent, or confidentiality of the data.

Regardless of how, when, or why they arise, limitations are a natural part of the research process and should never be ignored . Like all other aspects, they are vital in their own purpose.

Why is identifying limitations important?

Whether to seek acceptance or avoid struggle, humans often instinctively hide flaws and mistakes. Merging this thought process into research by attempting to hide limitations, however, is a bad idea. It has the potential to negate the validity of outcomes and damage the reputation of scholars.

By identifying and addressing limitations throughout a project, researchers strengthen their arguments and curtail the chance of peer censure based on overlooked mistakes. Pointing out these flaws shows an understanding of variable limits and a scrupulous research process.

Showing awareness of and taking responsibility for a project’s boundaries and challenges validates the integrity and transparency of a researcher. It further demonstrates the researchers understand the applicable literature and have thoroughly evaluated their chosen research methods.

Presenting limitations also benefits the readers by providing context for research findings. It guides them to interpret the project’s conclusions only within the scope of very specific conditions. By allowing for an appropriate generalization of the findings that is accurately confined by research boundaries and is not too broad, limitations boost a study’s credibility .

Limitations are true assets to the research process. They highlight opportunities for future research. When researchers identify the limitations of their particular approach to a study question, they enable precise transferability and improve chances for reproducibility. 

Simply stating a project’s limitations is not adequate for spurring further research, though. To spark the interest of other researchers, these acknowledgements must come with thorough explanations regarding how the limitations affected the current study and how they can potentially be overcome with amended methods.

How to write limitations

Typically, the information about a study’s limitations is situated either at the beginning of the discussion section to provide context for readers or at the conclusion of the discussion section to acknowledge the need for further research. However, it varies depending upon the target journal or publication guidelines. 

Don’t hide your limitations

It is also important to not bury a limitation in the body of the paper unless it has a unique connection to a topic in that section. If so, it needs to be reiterated with the other limitations or at the conclusion of the discussion section. Wherever it is included in the manuscript, ensure that the limitations section is prominently positioned and clearly introduced.

While maintaining transparency by disclosing limitations means taking a comprehensive approach, it is not necessary to discuss everything that could have potentially gone wrong during the research study. If there is no commitment to investigation in the introduction, it is unnecessary to consider the issue a limitation to the research. Wholly consider the term ‘limitations’ and ask, “Did it significantly change or limit the possible outcomes?” Then, qualify the occurrence as either a limitation to include in the current manuscript or as an idea to note for other projects. 

Writing limitations

Once the limitations are concretely identified and it is decided where they will be included in the paper, researchers are ready for the writing task. Including only what is pertinent, keeping explanations detailed but concise, and employing the following guidelines is key for crafting valuable limitations:

1) Identify and describe the limitations : Clearly introduce the limitation by classifying its form and specifying its origin. For example:

• An unintentional bias encountered during data collection
• An intentional use of unplanned post-hoc data analysis

2) Explain the implications : Describe how the limitation potentially influences the study’s findings and how the validity and generalizability are subsequently impacted. Provide examples and evidence to support claims of the limitations’ effects without making excuses or exaggerating their impact. Overall, be transparent and objective in presenting the limitations, without undermining the significance of the research. 

3) Provide alternative approaches for future studies : Offer specific suggestions for potential improvements or avenues for further investigation. Demonstrate a proactive approach by encouraging future research that addresses the identified gaps and, therefore, expands the knowledge base.

Whether presenting limitations as an individual section within the manuscript or as a subtopic in the discussion area, authors should use clear headings and straightforward language to facilitate readability. There is no need to complicate limitations with jargon, computations, or complex datasets.

Examples of common limitations

Limitations are generally grouped into two categories , methodology and research process .

Methodology limitations

Methodology may include limitations due to:

• Sample size
• Lack of available or reliable data
• Lack of prior research studies on the topic
• Measure used to collect the data
• Self-reported data

The researcher is addressing how the large sample size requires a reassessment of the measures used to collect and analyze the data.

Research process limitations

Limitations during the research process may arise from:

• Access to information
• Longitudinal effects
• Cultural and other biases
• Language fluency
• Time constraints

The author is pointing out that the model’s estimates are based on potentially biased observational studies.

Final thoughts

Successfully proving theories and touting great achievements are only two very narrow goals of scholarly research. The true passion and greatest efforts of researchers comes more in the form of confronting assumptions and exploring the obscure.

In many ways, recognizing and sharing the limitations of a research study both allows for and encourages this type of discovery that continuously pushes research forward. By using limitations to provide a transparent account of the project's boundaries and to contextualize the findings, researchers pave the way for even more robust and impactful research in the future.

Charla Viera, MS

See our "Privacy Policy"

Ensure your structure and ideas are consistent and clearly communicated

Pair your Premium Editing with our add-on service Presubmission Review for an overall assessment of your manuscript.

• Privacy Policy

Home » Limitations in Research – Types, Examples and Writing Guide

Limitations in Research – Types, Examples and Writing Guide

Table of Contents

Limitations in Research

Limitations in research refer to the factors that may affect the results, conclusions , and generalizability of a study. These limitations can arise from various sources, such as the design of the study, the sampling methods used, the measurement tools employed, and the limitations of the data analysis techniques.

Types of Limitations in Research

Types of Limitations in Research are as follows:

Sample Size Limitations

This refers to the size of the group of people or subjects that are being studied. If the sample size is too small, then the results may not be representative of the population being studied. This can lead to a lack of generalizability of the results.

Time Limitations

Time limitations can be a constraint on the research process . This could mean that the study is unable to be conducted for a long enough period of time to observe the long-term effects of an intervention, or to collect enough data to draw accurate conclusions.

Selection Bias

This refers to a type of bias that can occur when the selection of participants in a study is not random. This can lead to a biased sample that is not representative of the population being studied.

Confounding Variables

Confounding variables are factors that can influence the outcome of a study, but are not being measured or controlled for. These can lead to inaccurate conclusions or a lack of clarity in the results.

Measurement Error

This refers to inaccuracies in the measurement of variables, such as using a faulty instrument or scale. This can lead to inaccurate results or a lack of validity in the study.

Ethical Limitations

Ethical limitations refer to the ethical constraints placed on research studies. For example, certain studies may not be allowed to be conducted due to ethical concerns, such as studies that involve harm to participants.

Examples of Limitations in Research

Some Examples of Limitations in Research are as follows:

Research Title: “The Effectiveness of Machine Learning Algorithms in Predicting Customer Behavior”

Limitations:

• The study only considered a limited number of machine learning algorithms and did not explore the effectiveness of other algorithms.
• The study used a specific dataset, which may not be representative of all customer behaviors or demographics.
• The study did not consider the potential ethical implications of using machine learning algorithms in predicting customer behavior.

Research Title: “The Impact of Online Learning on Student Performance in Computer Science Courses”

• The study was conducted during the COVID-19 pandemic, which may have affected the results due to the unique circumstances of remote learning.
• The study only included students from a single university, which may limit the generalizability of the findings to other institutions.
• The study did not consider the impact of individual differences, such as prior knowledge or motivation, on student performance in online learning environments.

Research Title: “The Effect of Gamification on User Engagement in Mobile Health Applications”

• The study only tested a specific gamification strategy and did not explore the effectiveness of other gamification techniques.
• The study relied on self-reported measures of user engagement, which may be subject to social desirability bias or measurement errors.
• The study only included a specific demographic group (e.g., young adults) and may not be generalizable to other populations with different preferences or needs.

How to Write Limitations in Research

When writing about the limitations of a research study, it is important to be honest and clear about the potential weaknesses of your work. Here are some tips for writing about limitations in research:

• Identify the limitations: Start by identifying the potential limitations of your research. These may include sample size, selection bias, measurement error, or other issues that could affect the validity and reliability of your findings.
• Be honest and objective: When describing the limitations of your research, be honest and objective. Do not try to minimize or downplay the limitations, but also do not exaggerate them. Be clear and concise in your description of the limitations.
• Provide context: It is important to provide context for the limitations of your research. For example, if your sample size was small, explain why this was the case and how it may have affected your results. Providing context can help readers understand the limitations in a broader context.
• Discuss implications : Discuss the implications of the limitations for your research findings. For example, if there was a selection bias in your sample, explain how this may have affected the generalizability of your findings. This can help readers understand the limitations in terms of their impact on the overall validity of your research.
• Provide suggestions for future research : Finally, provide suggestions for future research that can address the limitations of your study. This can help readers understand how your research fits into the broader field and can provide a roadmap for future studies.

Purpose of Limitations in Research

There are several purposes of limitations in research. Here are some of the most important ones:

• To acknowledge the boundaries of the study : Limitations help to define the scope of the research project and set realistic expectations for the findings. They can help to clarify what the study is not intended to address.
• To identify potential sources of bias: Limitations can help researchers identify potential sources of bias in their research design, data collection, or analysis. This can help to improve the validity and reliability of the findings.
• To provide opportunities for future research: Limitations can highlight areas for future research and suggest avenues for further exploration. This can help to advance knowledge in a particular field.
• To demonstrate transparency and accountability: By acknowledging the limitations of their research, researchers can demonstrate transparency and accountability to their readers, peers, and funders. This can help to build trust and credibility in the research community.
• To encourage critical thinking: Limitations can encourage readers to critically evaluate the study’s findings and consider alternative explanations or interpretations. This can help to promote a more nuanced and sophisticated understanding of the topic under investigation.

When to Write Limitations in Research

Limitations should be included in research when they help to provide a more complete understanding of the study’s results and implications. A limitation is any factor that could potentially impact the accuracy, reliability, or generalizability of the study’s findings.

It is important to identify and discuss limitations in research because doing so helps to ensure that the results are interpreted appropriately and that any conclusions drawn are supported by the available evidence. Limitations can also suggest areas for future research, highlight potential biases or confounding factors that may have affected the results, and provide context for the study’s findings.

Generally, limitations should be discussed in the conclusion section of a research paper or thesis, although they may also be mentioned in other sections, such as the introduction or methods. The specific limitations that are discussed will depend on the nature of the study, the research question being investigated, and the data that was collected.

Examples of limitations that might be discussed in research include sample size limitations, data collection methods, the validity and reliability of measures used, and potential biases or confounding factors that could have affected the results. It is important to note that limitations should not be used as a justification for poor research design or methodology, but rather as a way to enhance the understanding and interpretation of the study’s findings.

Importance of Limitations in Research

Here are some reasons why limitations are important in research:

• Enhances the credibility of research: Limitations highlight the potential weaknesses and threats to validity, which helps readers to understand the scope and boundaries of the study. This improves the credibility of research by acknowledging its limitations and providing a clear picture of what can and cannot be concluded from the study.
• Facilitates replication: By highlighting the limitations, researchers can provide detailed information about the study’s methodology, data collection, and analysis. This information helps other researchers to replicate the study and test the validity of the findings, which enhances the reliability of research.
• Guides future research : Limitations provide insights into areas for future research by identifying gaps or areas that require further investigation. This can help researchers to design more comprehensive and effective studies that build on existing knowledge.
• Provides a balanced view: Limitations help to provide a balanced view of the research by highlighting both strengths and weaknesses. This ensures that readers have a clear understanding of the study’s limitations and can make informed decisions about the generalizability and applicability of the findings.

Advantages of Limitations in Research

Here are some potential advantages of limitations in research:

• Focus : Limitations can help researchers focus their study on a specific area or population, which can make the research more relevant and useful.
• Realism : Limitations can make a study more realistic by reflecting the practical constraints and challenges of conducting research in the real world.
• Innovation : Limitations can spur researchers to be more innovative and creative in their research design and methodology, as they search for ways to work around the limitations.
• Rigor : Limitations can actually increase the rigor and credibility of a study, as researchers are forced to carefully consider the potential sources of bias and error, and address them to the best of their abilities.
• Generalizability : Limitations can actually improve the generalizability of a study by ensuring that it is not overly focused on a specific sample or situation, and that the results can be applied more broadly.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

How to Cite Research Paper – All Formats and...

Data Collection – Methods Types and Examples

Delimitations in Research – Types, Examples and...

Research Paper Format – Types, Examples and...

Research Process – Steps, Examples and Tips

Research Design – Types, Methods and Examples

Research Limitations 101 📖

A Plain-Language Explainer (With Practical Examples)

By: Derek Jansen (MBA) | Expert Reviewer: Dr. Eunice Rautenbach | May 2024

Research limitations are one of those things that students tend to avoid digging into, and understandably so. No one likes to critique their own study and point out weaknesses. Nevertheless, being able to understand the limitations of your study – and, just as importantly, the implications thereof – a is a critically important skill.

In this post, we’ll unpack some of the most common research limitations you’re likely to encounter, so that you can approach your project with confidence.

Overview: Research Limitations 101

• What are research limitations ?
• Access – based limitations
• Temporal & financial limitations
• Sample & sampling limitations
• Design limitations
• Researcher limitations
• Key takeaways

What (exactly) are “research limitations”?

At the simplest level, research limitations (also referred to as “the limitations of the study”) are the constraints and challenges that will invariably influence your ability to conduct your study and draw reliable conclusions .

Research limitations are inevitable. Absolutely no study is perfect and limitations are an inherent part of any research design. These limitations can stem from a variety of sources , including access to data, methodological choices, and the more mundane constraints of budget and time. So, there’s no use trying to escape them – what matters is that you can recognise them.

Acknowledging and understanding these limitations is crucial, not just for the integrity of your research, but also for your development as a scholar. That probably sounds a bit rich, but realistically, having a strong understanding of the limitations of any given study helps you handle the inevitable obstacles professionally and transparently, which in turn builds trust with your audience and academic peers.

Simply put, recognising and discussing the limitations of your study demonstrates that you know what you’re doing , and that you’ve considered the results of your project within the context of these limitations. In other words, discussing the limitations is a sign of credibility and strength – not weakness. Contrary to the common misconception, highlighting your limitations (or rather, your study’s limitations) will earn you (rather than cost you) marks.

So, with that foundation laid, let’s have a look at some of the most common research limitations you’re likely to encounter – and how to go about managing them as effectively as possible.

Need a helping hand?

Limitation #1: Access To Information

One of the first hurdles you might encounter is limited access to necessary information. For example, you may have trouble getting access to specific literature or niche data sets. This situation can manifest due to several reasons, including paywalls, copyright and licensing issues or language barriers.

To minimise situations like these, it’s useful to try to leverage your university’s resource pool to the greatest extent possible. In practical terms, this means engaging with your university’s librarian and/or potentially utilising interlibrary loans to get access to restricted resources. If this sounds foreign to you, have a chat with your librarian 🙃

In emerging fields or highly specific study areas, you might find that there’s very little existing research (i.e., literature) on your topic. This scenario, while challenging, also offers a unique opportunity to contribute significantly to your field , as it indicates that there’s a significant research gap .

All of that said, be sure to conduct an exhaustive search using a variety of keywords and Boolean operators before assuming that there’s a lack of literature. Also, remember to snowball your literature base . In other words, scan the reference lists of the handful of papers that are directly relevant and then scan those references for more sources. You can also consider using tools like Litmaps and Connected Papers (see video below).

Limitation #2: Time & Money

Almost every researcher will face time and budget constraints at some point. Naturally, these limitations can affect the depth and breadth of your research – but they don’t need to be a death sentence.

Effective planning is crucial to managing both the temporal and financial aspects of your study. In practical terms, utilising tools like Gantt charts can help you visualise and plan your research timeline realistically, thereby reducing the risk of any nasty surprises. Always take a conservative stance when it comes to timelines, especially if you’re new to academic research. As a rule of thumb, things will generally take twice as long as you expect – so, prepare for the worst-case scenario.

If budget is a concern, you might want to consider exploring small research grants or adjusting the scope of your study so that it fits within a realistic budget. Trimming back might sound unattractive, but keep in mind that a smaller, well-planned study can often be more impactful than a larger, poorly planned project.

If you find yourself in a position where you’ve already run out of cash, don’t panic. There’s usually a pivot opportunity hidden somewhere within your project. Engage with your research advisor or faculty to explore potential solutions – don’t make any major changes without first consulting your institution.

Limitation #3: Sample Size & Composition

As we’ve discussed before , the size and representativeness of your sample are crucial , especially in quantitative research where the robustness of your conclusions often depends on these factors. All too often though, students run into issues achieving a sufficient sample size and composition.

To ensure adequacy in terms of your sample size, it’s important to plan for potential dropouts by oversampling from the outset . In other words, if you aim for a final sample size of 100 participants, aim to recruit 120-140 to account for unexpected challenges. If you still find yourself short on participants, consider whether you could complement your dataset with secondary data or data from an adjacent sample – for example, participants from another city or country. That said, be sure to engage with your research advisor before making any changes to your approach.

A related issue that you may run into is sample composition. In other words, you may have trouble securing a random sample that’s representative of your population of interest. In cases like this, you might again want to look at ways to complement your dataset with other sources, but if that’s not possible, it’s not the end of the world. As with all limitations, you’ll just need to recognise this limitation in your final write-up and be sure to interpret your results accordingly. In other words, don’t claim generalisability of your results if your sample isn’t random.

Limitation #4: Methodological Limitations

As we alluded earlier, every methodological choice comes with its own set of limitations . For example, you can’t claim causality if you’re using a descriptive or correlational research design. Similarly, as we saw in the previous example, you can’t claim generalisability if you’re using a non-random sampling approach.

Making good methodological choices is all about understanding (and accepting) the inherent trade-offs . In the vast majority of cases, you won’t be able to adopt the “perfect” methodology – and that’s okay. What’s important is that you select a methodology that aligns with your research aims and research questions , as well as the practical constraints at play (e.g., time, money, equipment access, etc.). Just as importantly, you must recognise and articulate the limitations of your chosen methods, and justify why they were the most suitable, given your specific context.

Limitation #5: Researcher (In)experience 

A discussion about research limitations would not be complete without mentioning the researcher (that’s you!). Whether we like to admit it or not, researcher inexperience and personal biases can subtly (and sometimes not so subtly) influence the interpretation and presentation of data within a study. This is especially true when it comes to dissertations and theses , as these are most commonly undertaken by first-time (or relatively fresh) researchers.

When it comes to dealing with this specific limitation, it’s important to remember the adage “ We don’t know what we don’t know ”. In other words, recognise and embrace your (relative) ignorance and subjectivity – and interpret your study’s results within that context . Simply put, don’t be overly confident in drawing conclusions from your study – especially when they contradict existing literature.

Cultivating a culture of reflexivity within your research practices can help reduce subjectivity and keep you a bit more “rooted” in the data. In practical terms, this simply means making an effort to become aware of how your perspectives and experiences may have shaped the research process and outcomes.

As with any new endeavour in life, it’s useful to garner as many outsider perspectives as possible. Of course, your university-assigned research advisor will play a large role in this respect, but it’s also a good idea to seek out feedback and critique from other academics. To this end, you might consider approaching other faculty at your institution, joining an online group, or even working with a private coach .

Key Takeaways

Understanding and effectively navigating research limitations is key to conducting credible and reliable academic work. By acknowledging and addressing these limitations upfront, you not only enhance the integrity of your research, but also demonstrate your academic maturity and professionalism.

Whether you’re working on a dissertation, thesis or any other type of formal academic research, remember the five most common research limitations and interpret your data while keeping them in mind.

• Access to Information (literature and data)
• Time and money
• Sample size and composition
• Research design and methodology
• Researcher (in)experience and bias

If you need a hand identifying and mitigating the limitations within your study, check out our 1:1 private coaching service .

Psst… there’s more!

This post is an extract from our bestselling short course, Methodology Bootcamp . If you want to work smart, you don't want to miss this .

You Might Also Like:

Submit a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

• Print Friendly

share this!

May 14, 2024

This article has been reviewed according to Science X's editorial process and policies . Editors have highlighted the following attributes while ensuring the content's credibility:

fact-checked

trusted source

written by researcher(s)

Most crime has fallen by 90% in 30 years—so why does the public think it's increased?

by Toby Davies and Graham Farrell, The Conversation

Seventy-eight percent of people in England and Wales think that crime has gone up in the last few years, according to the latest survey . But the data on actual crime shows the exact opposite.

As of 2024, violence, burglary and car crime have been declining for 30 years and by close to 90%, according to the Crime Survey for England and Wales (CSEW)—our best indicator of true crime levels. Unlike police data, the CSEW is not subject to variations in reporting and recording.

The drop in violence includes domestic violence and other violence against women. Anti-social behavior has similarly declined. While increased fraud and computer misuse now make up half of crime , this mainly reflects how far the rates of other crimes have fallen.

All high-income countries have experienced similar trends, and there is scientific consensus that the decline in crime is a real phenomenon .

There is strong research evidence that security improvements were responsible for the drop. This is most obvious with vehicle electronic immobilizers and door deadlocks, and better household security —stronger door frames, double glazed windows and security fittings—along with an avalanche of security in shopping centers, sports stadiums, schools, businesses and elsewhere. Quite simply, it became more difficult to commit crimes.

Decreases in crimes often committed by teenagers, such as joyriding or burglary, had a multiplying effect: when teenagers could no longer commit these easy "debut crimes" they did not progress to longer criminal careers.

There are, of course, exceptions. Some places, times and crime types had a less pronounced decline or even an increase. For many years, phone theft was an exception to the general decline in theft. Cybercrime, measured by the CSEW as fraud and computer misuse, has increased and is the most prominent exception.

But this increase was not due to thwarted burglars and car thieves switching targets: the skillset, resources and rewards for cybercrime are very different . Rather, it reflects new crime opportunities facilitated by the internet. Preventive policy and practice is slowly getting better at closing off opportunities for computer misuse, but work is needed to accelerate those prevention efforts.

The perception gap

So why is there such a gulf between public perception and the reality of crime trends? A regular YouGov poll asks respondents for their top three concerns from a broad set of issues. Concern about crime went from a low in 2016 (when people were more concerned with Brexit), quadrupled by 2019 and plummeted during the pandemic when people had other worries. But in the last year, the public's concern about crime has risen again.

Proportion of people naming crime as a top three issue facing the country:

There are many possible explanations for this, of which the first is poor information. A study published in 1998 found that "people who watch a lot of television or who read a lot of newspapers will be exposed to a steady diet of crime stories" that does not reflect official statistics.

The old news media adage "if it bleeds, it leads" reflects how violent news stories, including crime increases and serious crimes, capture public attention. Knife crime grabs headlines in the UK, but our shock at individual incidents is testament to their rarity and our relative success in controlling violence—many gun crimes do not make the news in the US.

Most recent terrorist attacks in the UK have featured knives (plus a thwarted Liverpool bomber ), but there is little discussion of how this indicates that measures to restrict guns and bomb-making resources are effective.

Political rhetoric can also skew perceptions, particularly in the run-up to elections. During the recent local elections, the Conservatives were widely criticized for an advert portraying London as "a crime capital of the world" (using a video of New York), while Labor has also made reference to high levels of crime under the current government.

There are also some "crime drop deniers," who have vested interests in crime not declining due to, for example, fear of budget cuts. One of us (Graham) worked with a former police chief who routinely denied the existence of declining crime.

Despite the evidence of crime rates dropping, some concerns are justified. Victims, along with their families and friends, have legitimate concerns, particularly as crime is more likely to recur against the same people and at the same places.

And, while the trend is clear, there are nevertheless localized increases in some types of offending. When these relate to harmful and emotive issues like knife crime in London, for example, it is natural that this might have a substantial influence.

We are unlikely to be able to change political agendas or journalists' approach to reporting. But governments should be taking a more rational approach to crime that is based on evidence, not public perception.

Local governments need to keep on top of their local crime hotspots: problem bars and clubs where crime occurs, shops where shoplifting is concentrated, local road traffic offense hotspots and so on. The common theme here is how crime concentrates.

National government, meanwhile, should lead on reducing crime opportunities via national-level levers. Only national government can influence social media platforms and websites that host online crime and encourage larger businesses to improve manufacturing, retailing and service industry practices.

The positive story around crime rarely makes headlines, but this should not put us off from learning the lessons borne out in the data. We know this can work from past success, but it took decades to get car makers to improve vehicle security and to get secure-by-design ideas in building regulations. Society needs to move more quickly.

Provided by The Conversation

Explore further

Feedback to editors

Scientists develop new geochemical 'fingerprint' to trace contaminants in fertilizer

7 hours ago

Study reveals how a sugar-sensing protein acts as a 'machine' to switch plant growth—and oil production—on and off

8 hours ago

Researchers develop world's smallest quantum light detector on a silicon chip

How heat waves are affecting Arctic phytoplankton

Horse remains show Pagan-Christian trade networks supplied horses from overseas for the last horse sacrifices in Europe

Ion irradiation offers promise for 2D material probing

Furry thieves are running loose in a Maine forest, research shows

Study indicates Earth's earliest sea creatures drove evolution by stirring the water

9 hours ago

Chemists develop new method for introducing fluorinated components into molecules

From fungi to fashion: Mushroom eco-leather is moving towards the mainstream

Relevant physicsforums posts, cover songs versus the original track, which ones are better, what are your favorite disco "classics".

16 hours ago

Who is your favorite Jazz musician and what is your favorite song?

20 hours ago

For WW2 buffs!

21 hours ago

Blues and Broken Hearts

May 16, 2024

Which ancient civilizations are you most interested in?

May 15, 2024

More from Art, Music, History, and Linguistics

Related Stories

Areas with more illegal market opportunities more likely to be targeted by organized crime, study shows

May 9, 2024

Research investigates whether electing progressive US prosecutors leads to increased crime

Apr 22, 2024

Crime detection and crime hot spot prediction using a deep learning model

Apr 19, 2024

More accurately capturing data on crime using surveys

Feb 21, 2022

Virtual reality study finds that environmental interventions may not reduce fear of crime

Feb 8, 2024

Finding a better way to use cameras to reduce crime

Dec 12, 2023

Recommended for you

Military rank affects medical care, offering societal insights: Study

Study finds saying 'please' may not be so polite in everyday requests

Singing researchers find cross-cultural patterns in music and language

Mechanistic model shows how much gossip is needed to foster social cooperation

The power of ambiguity: Using computer models to understand the debate about climate change

May 13, 2024

Study finds avoiding social media before an election has little to no effect on people's political views

Let us know if there is a problem with our content.

Use this form if you have come across a typo, inaccuracy or would like to send an edit request for the content on this page. For general inquiries, please use our contact form . For general feedback, use the public comments section below (please adhere to guidelines ).

Please select the most appropriate category to facilitate processing of your request

Thank you for taking time to provide your feedback to the editors.

Your feedback is important to us. However, we do not guarantee individual replies due to the high volume of messages.

E-mail the story

Your email address is used only to let the recipient know who sent the email. Neither your address nor the recipient's address will be used for any other purpose. The information you enter will appear in your e-mail message and is not retained by Phys.org in any form.

Newsletter sign up

Get weekly and/or daily updates delivered to your inbox. You can unsubscribe at any time and we'll never share your details to third parties.

More information Privacy policy

Donate and enjoy an ad-free experience

We keep our content available to everyone. Consider supporting Science X's mission by getting a premium account.

E-mail newsletter

Cultural Relativity and Acceptance of Embryonic Stem Cell Research

Article sidebar.

Main Article Content

There is a debate about the ethical implications of using human embryos in stem cell research, which can be influenced by cultural, moral, and social values. This paper argues for an adaptable framework to accommodate diverse cultural and religious perspectives. By using an adaptive ethics model, research protections can reflect various populations and foster growth in stem cell research possibilities.

INTRODUCTION

Stem cell research combines biology, medicine, and technology, promising to alter health care and the understanding of human development. Yet, ethical contention exists because of individuals’ perceptions of using human embryos based on their various cultural, moral, and social values. While these disagreements concerning policy, use, and general acceptance have prompted the development of an international ethics policy, such a uniform approach can overlook the nuanced ethical landscapes between cultures. With diverse viewpoints in public health, a single global policy, especially one reflecting Western ethics or the ethics prevalent in high-income countries, is impractical. This paper argues for a culturally sensitive, adaptable framework for the use of embryonic stem cells. Stem cell policy should accommodate varying ethical viewpoints and promote an effective global dialogue. With an extension of an ethics model that can adapt to various cultures, we recommend localized guidelines that reflect the moral views of the people those guidelines serve.

Stem cells, characterized by their unique ability to differentiate into various cell types, enable the repair or replacement of damaged tissues. Two primary types of stem cells are somatic stem cells (adult stem cells) and embryonic stem cells. Adult stem cells exist in developed tissues and maintain the body’s repair processes. [1] Embryonic stem cells (ESC) are remarkably pluripotent or versatile, making them valuable in research. [2] However, the use of ESCs has sparked ethics debates. Considering the potential of embryonic stem cells, research guidelines are essential. The International Society for Stem Cell Research (ISSCR) provides international stem cell research guidelines. They call for “public conversations touching on the scientific significance as well as the societal and ethical issues raised by ESC research.” [3] The ISSCR also publishes updates about culturing human embryos 14 days post fertilization, suggesting local policies and regulations should continue to evolve as ESC research develops. [4]  Like the ISSCR, which calls for local law and policy to adapt to developing stem cell research given cultural acceptance, this paper highlights the importance of local social factors such as religion and culture.

I.     Global Cultural Perspective of Embryonic Stem Cells

Views on ESCs vary throughout the world. Some countries readily embrace stem cell research and therapies, while others have stricter regulations due to ethical concerns surrounding embryonic stem cells and when an embryo becomes entitled to moral consideration. The philosophical issue of when the “someone” begins to be a human after fertilization, in the morally relevant sense, [5] impacts when an embryo becomes not just worthy of protection but morally entitled to it. The process of creating embryonic stem cell lines involves the destruction of the embryos for research. [6] Consequently, global engagement in ESC research depends on social-cultural acceptability.

a.     US and Rights-Based Cultures

In the United States, attitudes toward stem cell therapies are diverse. The ethics and social approaches, which value individualism, [7] trigger debates regarding the destruction of human embryos, creating a complex regulatory environment. For example, the 1996 Dickey-Wicker Amendment prohibited federal funding for the creation of embryos for research and the destruction of embryos for “more than allowed for research on fetuses in utero.” [8] Following suit, in 2001, the Bush Administration heavily restricted stem cell lines for research. However, the Stem Cell Research Enhancement Act of 2005 was proposed to help develop ESC research but was ultimately vetoed. [9] Under the Obama administration, in 2009, an executive order lifted restrictions allowing for more development in this field. [10] The flux of research capacity and funding parallels the different cultural perceptions of human dignity of the embryo and how it is socially presented within the country’s research culture. [11]

b.     Ubuntu and Collective Cultures

African bioethics differs from Western individualism because of the different traditions and values. African traditions, as described by individuals from South Africa and supported by some studies in other African countries, including Ghana and Kenya, follow the African moral philosophies of Ubuntu or Botho and Ukama , which “advocates for a form of wholeness that comes through one’s relationship and connectedness with other people in the society,” [12] making autonomy a socially collective concept. In this context, for the community to act autonomously, individuals would come together to decide what is best for the collective. Thus, stem cell research would require examining the value of the research to society as a whole and the use of the embryos as a collective societal resource. If society views the source as part of the collective whole, and opposes using stem cells, compromising the cultural values to pursue research may cause social detachment and stunt research growth. [13] Based on local culture and moral philosophy, the permissibility of stem cell research depends on how embryo, stem cell, and cell line therapies relate to the community as a whole. Ubuntu is the expression of humanness, with the person’s identity drawn from the “’I am because we are’” value. [14] The decision in a collectivistic culture becomes one born of cultural context, and individual decisions give deference to others in the society.

Consent differs in cultures where thought and moral philosophy are based on a collective paradigm. So, applying Western bioethical concepts is unrealistic. For one, Africa is a diverse continent with many countries with different belief systems, access to health care, and reliance on traditional or Western medicines. Where traditional medicine is the primary treatment, the “’restrictive focus on biomedically-related bioethics’” [is] problematic in African contexts because it neglects bioethical issues raised by traditional systems.” [15] No single approach applies in all areas or contexts. Rather than evaluating the permissibility of ESC research according to Western concepts such as the four principles approach, different ethics approaches should prevail.

Another consideration is the socio-economic standing of countries. In parts of South Africa, researchers have not focused heavily on contributing to the stem cell discourse, either because it is not considered health care or a health science priority or because resources are unavailable. [16] Each country’s priorities differ given different social, political, and economic factors. In South Africa, for instance, areas such as maternal mortality, non-communicable diseases, telemedicine, and the strength of health systems need improvement and require more focus. [17] Stem cell research could benefit the population, but it also could divert resources from basic medical care. Researchers in South Africa adhere to the National Health Act and Medicines Control Act in South Africa and international guidelines; however, the Act is not strictly enforced, and there is no clear legislation for research conduct or ethical guidelines. [18]

Some parts of Africa condemn stem cell research. For example, 98.2 percent of the Tunisian population is Muslim. [19] Tunisia does not permit stem cell research because of moral conflict with a Fatwa. Religion heavily saturates the regulation and direction of research. [20] Stem cell use became permissible for reproductive purposes only recently, with tight restrictions preventing cells from being used in any research other than procedures concerning ART/IVF.  Their use is conditioned on consent, and available only to married couples. [21] The community's receptiveness to stem cell research depends on including communitarian African ethics.

c.     Asia

Some Asian countries also have a collective model of ethics and decision making. [22] In China, the ethics model promotes a sincere respect for life or human dignity, [23] based on protective medicine. This model, influenced by Traditional Chinese Medicine (TCM), [24] recognizes Qi as the vital energy delivered via the meridians of the body; it connects illness to body systems, the body’s entire constitution, and the universe for a holistic bond of nature, health, and quality of life. [25] Following a protective ethics model, and traditional customs of wholeness, investment in stem cell research is heavily desired for its applications in regenerative therapies, disease modeling, and protective medicines. In a survey of medical students and healthcare practitioners, 30.8 percent considered stem cell research morally unacceptable while 63.5 percent accepted medical research using human embryonic stem cells. Of these individuals, 89.9 percent supported increased funding for stem cell research. [26] The scientific community might not reflect the overall population. From 1997 to 2019, China spent a total of $576 million (USD) on stem cell research at 8,050 stem cell programs, increased published presence from 0.6 percent to 14.01 percent of total global stem cell publications as of 2014, and made significant strides in cell-based therapies for various medical conditions. [27] However, while China has made substantial investments in stem cell research and achieved notable progress in clinical applications, concerns linger regarding ethical oversight and transparency. [28] For example, the China Biosecurity Law, promoted by the National Health Commission and China Hospital Association, attempted to mitigate risks by introducing an institutional review board (IRB) in the regulatory bodies. 5800 IRBs registered with the Chinese Clinical Trial Registry since 2021. [29] However, issues still need to be addressed in implementing effective IRB review and approval procedures.

The substantial government funding and focus on scientific advancement have sometimes overshadowed considerations of regional cultures, ethnic minorities, and individual perspectives, particularly evident during the one-child policy era. As government policy adapts to promote public stability, such as the change from the one-child to the two-child policy, [30] research ethics should also adapt to ensure respect for the values of its represented peoples.

Japan is also relatively supportive of stem cell research and therapies. Japan has a more transparent regulatory framework, allowing for faster approval of regenerative medicine products, which has led to several advanced clinical trials and therapies. [31] South Korea is also actively engaged in stem cell research and has a history of breakthroughs in cloning and embryonic stem cells. [32] However, the field is controversial, and there are issues of scientific integrity. For example, the Korean FDA fast-tracked products for approval, [33] and in another instance, the oocyte source was unclear and possibly violated ethical standards. [34] Trust is important in research, as it builds collaborative foundations between colleagues, trial participant comfort, open-mindedness for complicated and sensitive discussions, and supports regulatory procedures for stakeholders. There is a need to respect the culture’s interest, engagement, and for research and clinical trials to be transparent and have ethical oversight to promote global research discourse and trust.

d.     Middle East

Countries in the Middle East have varying degrees of acceptance of or restrictions to policies related to using embryonic stem cells due to cultural and religious influences. Saudi Arabia has made significant contributions to stem cell research, and conducts research based on international guidelines for ethical conduct and under strict adherence to guidelines in accordance with Islamic principles. Specifically, the Saudi government and people require ESC research to adhere to Sharia law. In addition to umbilical and placental stem cells, [35] Saudi Arabia permits the use of embryonic stem cells as long as they come from miscarriages, therapeutic abortions permissible by Sharia law, or are left over from in vitro fertilization and donated to research. [36] Laws and ethical guidelines for stem cell research allow the development of research institutions such as the King Abdullah International Medical Research Center, which has a cord blood bank and a stem cell registry with nearly 10,000 donors. [37] Such volume and acceptance are due to the ethical ‘permissibility’ of the donor sources, which do not conflict with religious pillars. However, some researchers err on the side of caution, choosing not to use embryos or fetal tissue as they feel it is unethical to do so. [38]

Jordan has a positive research ethics culture. [39] However, there is a significant issue of lack of trust in researchers, with 45.23 percent (38.66 percent agreeing and 6.57 percent strongly agreeing) of Jordanians holding a low level of trust in researchers, compared to 81.34 percent of Jordanians agreeing that they feel safe to participate in a research trial. [40] Safety testifies to the feeling of confidence that adequate measures are in place to protect participants from harm, whereas trust in researchers could represent the confidence in researchers to act in the participants’ best interests, adhere to ethical guidelines, provide accurate information, and respect participants’ rights and dignity. One method to improve trust would be to address communication issues relevant to ESC. Legislation surrounding stem cell research has adopted specific language, especially concerning clarification “between ‘stem cells’ and ‘embryonic stem cells’” in translation. [41] Furthermore, legislation “mandates the creation of a national committee… laying out specific regulations for stem-cell banking in accordance with international standards.” [42] This broad regulation opens the door for future global engagement and maintains transparency. However, these regulations may also constrain the influence of research direction, pace, and accessibility of research outcomes.

e.     Europe

In the European Union (EU), ethics is also principle-based, but the principles of autonomy, dignity, integrity, and vulnerability are interconnected. [43] As such, the opportunity for cohesion and concessions between individuals’ thoughts and ideals allows for a more adaptable ethics model due to the flexible principles that relate to the human experience The EU has put forth a framework in its Convention for the Protection of Human Rights and Dignity of the Human Being allowing member states to take different approaches. Each European state applies these principles to its specific conventions, leading to or reflecting different acceptance levels of stem cell research. [44]

For example, in Germany, Lebenzusammenhang , or the coherence of life, references integrity in the unity of human culture. Namely, the personal sphere “should not be subject to external intervention.” [45]  Stem cell interventions could affect this concept of bodily completeness, leading to heavy restrictions. Under the Grundgesetz, human dignity and the right to life with physical integrity are paramount. [46] The Embryo Protection Act of 1991 made producing cell lines illegal. Cell lines can be imported if approved by the Central Ethics Commission for Stem Cell Research only if they were derived before May 2007. [47] Stem cell research respects the integrity of life for the embryo with heavy specifications and intense oversight. This is vastly different in Finland, where the regulatory bodies find research more permissible in IVF excess, but only up to 14 days after fertilization. [48] Spain’s approach differs still, with a comprehensive regulatory framework. [49] Thus, research regulation can be culture-specific due to variations in applied principles. Diverse cultures call for various approaches to ethical permissibility. [50] Only an adaptive-deliberative model can address the cultural constructions of self and achieve positive, culturally sensitive stem cell research practices. [51]

II.     Religious Perspectives on ESC

Embryonic stem cell sources are the main consideration within religious contexts. While individuals may not regard their own religious texts as authoritative or factual, religion can shape their foundations or perspectives.

The Qur'an states:

“And indeed We created man from a quintessence of clay. Then We placed within him a small quantity of nutfa (sperm to fertilize) in a safe place. Then We have fashioned the nutfa into an ‘alaqa (clinging clot or cell cluster), then We developed the ‘alaqa into mudgha (a lump of flesh), and We made mudgha into bones, and clothed the bones with flesh, then We brought it into being as a new creation. So Blessed is Allah, the Best of Creators.” [52]

Many scholars of Islam estimate the time of soul installment, marked by the angel breathing in the soul to bring the individual into creation, as 120 days from conception. [53] Personhood begins at this point, and the value of life would prohibit research or experimentation that could harm the individual. If the fetus is more than 120 days old, the time ensoulment is interpreted to occur according to Islamic law, abortion is no longer permissible. [54] There are a few opposing opinions about early embryos in Islamic traditions. According to some Islamic theologians, there is no ensoulment of the early embryo, which is the source of stem cells for ESC research. [55]

In Buddhism, the stance on stem cell research is not settled. The main tenets, the prohibition against harming or destroying others (ahimsa) and the pursuit of knowledge (prajña) and compassion (karuna), leave Buddhist scholars and communities divided. [56] Some scholars argue stem cell research is in accordance with the Buddhist tenet of seeking knowledge and ending human suffering. Others feel it violates the principle of not harming others. Finding the balance between these two points relies on the karmic burden of Buddhist morality. In trying to prevent ahimsa towards the embryo, Buddhist scholars suggest that to comply with Buddhist tenets, research cannot be done as the embryo has personhood at the moment of conception and would reincarnate immediately, harming the individual's ability to build their karmic burden. [57] On the other hand, the Bodhisattvas, those considered to be on the path to enlightenment or Nirvana, have given organs and flesh to others to help alleviate grieving and to benefit all. [58] Acceptance varies on applied beliefs and interpretations.

Catholicism does not support embryonic stem cell research, as it entails creation or destruction of human embryos. This destruction conflicts with the belief in the sanctity of life. For example, in the Old Testament, Genesis describes humanity as being created in God’s image and multiplying on the Earth, referencing the sacred rights to human conception and the purpose of development and life. In the Ten Commandments, the tenet that one should not kill has numerous interpretations where killing could mean murder or shedding of the sanctity of life, demonstrating the high value of human personhood. In other books, the theological conception of when life begins is interpreted as in utero, [59] highlighting the inviolability of life and its formation in vivo to make a religious point for accepting such research as relatively limited, if at all. [60] The Vatican has released ethical directives to help apply a theological basis to modern-day conflicts. The Magisterium of the Church states that “unless there is a moral certainty of not causing harm,” experimentation on fetuses, fertilized cells, stem cells, or embryos constitutes a crime. [61] Such procedures would not respect the human person who exists at these stages, according to Catholicism. Damages to the embryo are considered gravely immoral and illicit. [62] Although the Catholic Church officially opposes abortion, surveys demonstrate that many Catholic people hold pro-choice views, whether due to the context of conception, stage of pregnancy, threat to the mother’s life, or for other reasons, demonstrating that practicing members can also accept some but not all tenets. [63]

Some major Jewish denominations, such as the Reform, Conservative, and Reconstructionist movements, are open to supporting ESC use or research as long as it is for saving a life. [64] Within Judaism, the Talmud, or study, gives personhood to the child at birth and emphasizes that life does not begin at conception: [65]

“If she is found pregnant, until the fortieth day it is mere fluid,” [66]

Whereas most religions prioritize the status of human embryos, the Halakah (Jewish religious law) states that to save one life, most other religious laws can be ignored because it is in pursuit of preservation. [67] Stem cell research is accepted due to application of these religious laws.

We recognize that all religions contain subsets and sects. The variety of environmental and cultural differences within religious groups requires further analysis to respect the flexibility of religious thoughts and practices. We make no presumptions that all cultures require notions of autonomy or morality as under the common morality theory , which asserts a set of universal moral norms that all individuals share provides moral reasoning and guides ethical decisions. [68] We only wish to show that the interaction with morality varies between cultures and countries.

III.     A Flexible Ethical Approach

The plurality of different moral approaches described above demonstrates that there can be no universally acceptable uniform law for ESC on a global scale. Instead of developing one standard, flexible ethical applications must be continued. We recommend local guidelines that incorporate important cultural and ethical priorities.

While the Declaration of Helsinki is more relevant to people in clinical trials receiving ESC products, in keeping with the tradition of protections for research subjects, consent of the donor is an ethical requirement for ESC donation in many jurisdictions including the US, Canada, and Europe. [69] The Declaration of Helsinki provides a reference point for regulatory standards and could potentially be used as a universal baseline for obtaining consent prior to gamete or embryo donation.

For instance, in Columbia University’s egg donor program for stem cell research, donors followed standard screening protocols and “underwent counseling sessions that included information as to the purpose of oocyte donation for research, what the oocytes would be used for, the risks and benefits of donation, and process of oocyte stimulation” to ensure transparency for consent. [70] The program helped advance stem cell research and provided clear and safe research methods with paid participants. Though paid participation or covering costs of incidental expenses may not be socially acceptable in every culture or context, [71] and creating embryos for ESC research is illegal in many jurisdictions, Columbia’s program was effective because of the clear and honest communications with donors, IRBs, and related stakeholders.  This example demonstrates that cultural acceptance of scientific research and of the idea that an egg or embryo does not have personhood is likely behind societal acceptance of donating eggs for ESC research. As noted, many countries do not permit the creation of embryos for research.

Proper communication and education regarding the process and purpose of stem cell research may bolster comprehension and garner more acceptance. “Given the sensitive subject material, a complete consent process can support voluntary participation through trust, understanding, and ethical norms from the cultures and morals participants value. This can be hard for researchers entering countries of different socioeconomic stability, with different languages and different societal values. [72]

An adequate moral foundation in medical ethics is derived from the cultural and religious basis that informs knowledge and actions. [73] Understanding local cultural and religious values and their impact on research could help researchers develop humility and promote inclusion.

IV.     Concerns

Some may argue that if researchers all adhere to one ethics standard, protection will be satisfied across all borders, and the global public will trust researchers. However, defining what needs to be protected and how to define such research standards is very specific to the people to which standards are applied. We suggest that applying one uniform guide cannot accurately protect each individual because we all possess our own perceptions and interpretations of social values. [74] Therefore, the issue of not adjusting to the moral pluralism between peoples in applying one standard of ethics can be resolved by building out ethics models that can be adapted to different cultures and religions.

Other concerns include medical tourism, which may promote health inequities. [75] Some countries may develop and approve products derived from ESC research before others, compromising research ethics or drug approval processes. There are also concerns about the sale of unauthorized stem cell treatments, for example, those without FDA approval in the United States. Countries with robust research infrastructures may be tempted to attract medical tourists, and some customers will have false hopes based on aggressive publicity of unproven treatments. [76]

For example, in China, stem cell clinics can market to foreign clients who are not protected under the regulatory regimes. Companies employ a marketing strategy of “ethically friendly” therapies. Specifically, in the case of Beike, China’s leading stem cell tourism company and sprouting network, ethical oversight of administrators or health bureaus at one site has “the unintended consequence of shifting questionable activities to another node in Beike's diffuse network.” [77] In contrast, Jordan is aware of stem cell research’s potential abuse and its own status as a “health-care hub.” Jordan’s expanded regulations include preserving the interests of individuals in clinical trials and banning private companies from ESC research to preserve transparency and the integrity of research practices. [78]

The social priorities of the community are also a concern. The ISSCR explicitly states that guidelines “should be periodically revised to accommodate scientific advances, new challenges, and evolving social priorities.” [79] The adaptable ethics model extends this consideration further by addressing whether research is warranted given the varying degrees of socioeconomic conditions, political stability, and healthcare accessibilities and limitations. An ethical approach would require discussion about resource allocation and appropriate distribution of funds. [80]

While some religions emphasize the sanctity of life from conception, which may lead to public opposition to ESC research, others encourage ESC research due to its potential for healing and alleviating human pain. Many countries have special regulations that balance local views on embryonic personhood, the benefits of research as individual or societal goods, and the protection of human research subjects. To foster understanding and constructive dialogue, global policy frameworks should prioritize the protection of universal human rights, transparency, and informed consent. In addition to these foundational global policies, we recommend tailoring local guidelines to reflect the diverse cultural and religious perspectives of the populations they govern. Ethics models should be adapted to local populations to effectively establish research protections, growth, and possibilities of stem cell research.

For example, in countries with strong beliefs in the moral sanctity of embryos or heavy religious restrictions, an adaptive model can allow for discussion instead of immediate rejection. In countries with limited individual rights and voice in science policy, an adaptive model ensures cultural, moral, and religious views are taken into consideration, thereby building social inclusion. While this ethical consideration by the government may not give a complete voice to every individual, it will help balance policies and maintain the diverse perspectives of those it affects. Embracing an adaptive ethics model of ESC research promotes open-minded dialogue and respect for the importance of human belief and tradition. By actively engaging with cultural and religious values, researchers can better handle disagreements and promote ethical research practices that benefit each society.

This brief exploration of the religious and cultural differences that impact ESC research reveals the nuances of relative ethics and highlights a need for local policymakers to apply a more intense adaptive model.

[1] Poliwoda, S., Noor, N., Downs, E., Schaaf, A., Cantwell, A., Ganti, L., Kaye, A. D., Mosel, L. I., Carroll, C. B., Viswanath, O., & Urits, I. (2022). Stem cells: a comprehensive review of origins and emerging clinical roles in medical practice.  Orthopedic reviews ,  14 (3), 37498. https://doi.org/10.52965/001c.37498

[2] Poliwoda, S., Noor, N., Downs, E., Schaaf, A., Cantwell, A., Ganti, L., Kaye, A. D., Mosel, L. I., Carroll, C. B., Viswanath, O., & Urits, I. (2022). Stem cells: a comprehensive review of origins and emerging clinical roles in medical practice.  Orthopedic reviews ,  14 (3), 37498. https://doi.org/10.52965/001c.37498

[3] International Society for Stem Cell Research. (2023). Laboratory-based human embryonic stem cell research, embryo research, and related research activities . International Society for Stem Cell Research. https://www.isscr.org/guidelines/blog-post-title-one-ed2td-6fcdk ; Kimmelman, J., Hyun, I., Benvenisty, N.  et al.  Policy: Global standards for stem-cell research.  Nature   533 , 311–313 (2016). https://doi.org/10.1038/533311a

[4] International Society for Stem Cell Research. (2023). Laboratory-based human embryonic stem cell research, embryo research, and related research activities . International Society for Stem Cell Research. https://www.isscr.org/guidelines/blog-post-title-one-ed2td-6fcdk

[5] Concerning the moral philosophies of stem cell research, our paper does not posit a personal moral stance nor delve into the “when” of human life begins. To read further about the philosophical debate, consider the following sources:

Sandel M. J. (2004). Embryo ethics--the moral logic of stem-cell research.  The New England journal of medicine ,  351 (3), 207–209. https://doi.org/10.1056/NEJMp048145 ; George, R. P., & Lee, P. (2020, September 26). Acorns and Embryos . The New Atlantis. https://www.thenewatlantis.com/publications/acorns-and-embryos ; Sagan, A., & Singer, P. (2007). The moral status of stem cells. Metaphilosophy , 38 (2/3), 264–284. http://www.jstor.org/stable/24439776 ; McHugh P. R. (2004). Zygote and "clonote"--the ethical use of embryonic stem cells.  The New England journal of medicine ,  351 (3), 209–211. https://doi.org/10.1056/NEJMp048147 ; Kurjak, A., & Tripalo, A. (2004). The facts and doubts about beginning of the human life and personality.  Bosnian journal of basic medical sciences ,  4 (1), 5–14. https://doi.org/10.17305/bjbms.2004.3453

[6] Vazin, T., & Freed, W. J. (2010). Human embryonic stem cells: derivation, culture, and differentiation: a review.  Restorative neurology and neuroscience ,  28 (4), 589–603. https://doi.org/10.3233/RNN-2010-0543

[7] Socially, at its core, the Western approach to ethics is widely principle-based, autonomy being one of the key factors to ensure a fundamental respect for persons within research. For information regarding autonomy in research, see: Department of Health, Education, and Welfare, & National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research (1978). The Belmont Report. Ethical principles and guidelines for the protection of human subjects of research.; For a more in-depth review of autonomy within the US, see: Beauchamp, T. L., & Childress, J. F. (1994). Principles of Biomedical Ethics . Oxford University Press.

[8] Sherley v. Sebelius , 644 F.3d 388 (D.C. Cir. 2011), citing 45 C.F.R. 46.204(b) and [42 U.S.C. § 289g(b)]. https://www.cadc.uscourts.gov/internet/opinions.nsf/6c690438a9b43dd685257a64004ebf99/$file/11-5241-1391178.pdf

[9] Stem Cell Research Enhancement Act of 2005, H. R. 810, 109 th Cong. (2001). https://www.govtrack.us/congress/bills/109/hr810/text ; Bush, G. W. (2006, July 19). Message to the House of Representatives . National Archives and Records Administration. https://georgewbush-whitehouse.archives.gov/news/releases/2006/07/20060719-5.html

[10] National Archives and Records Administration. (2009, March 9). Executive order 13505 -- removing barriers to responsible scientific research involving human stem cells . National Archives and Records Administration. https://obamawhitehouse.archives.gov/the-press-office/removing-barriers-responsible-scientific-research-involving-human-stem-cells

[11] Hurlbut, W. B. (2006). Science, Religion, and the Politics of Stem Cells.  Social Research ,  73 (3), 819–834. http://www.jstor.org/stable/40971854

[12] Akpa-Inyang, Francis & Chima, Sylvester. (2021). South African traditional values and beliefs regarding informed consent and limitations of the principle of respect for autonomy in African communities: a cross-cultural qualitative study. BMC Medical Ethics . 22. 10.1186/s12910-021-00678-4.

[13] Source for further reading: Tangwa G. B. (2007). Moral status of embryonic stem cells: perspective of an African villager. Bioethics , 21(8), 449–457. https://doi.org/10.1111/j.1467-8519.2007.00582.x , see also Mnisi, F. M. (2020). An African analysis based on ethics of Ubuntu - are human embryonic stem cell patents morally justifiable? African Insight , 49 (4).

[14] Jecker, N. S., & Atuire, C. (2021). Bioethics in Africa: A contextually enlightened analysis of three cases. Developing World Bioethics , 22 (2), 112–122. https://doi.org/10.1111/dewb.12324

[15] Jecker, N. S., & Atuire, C. (2021). Bioethics in Africa: A contextually enlightened analysis of three cases. Developing World Bioethics, 22(2), 112–122. https://doi.org/10.1111/dewb.12324

[16] Jackson, C.S., Pepper, M.S. Opportunities and barriers to establishing a cell therapy programme in South Africa.  Stem Cell Res Ther   4 , 54 (2013). https://doi.org/10.1186/scrt204 ; Pew Research Center. (2014, May 1). Public health a major priority in African nations . Pew Research Center’s Global Attitudes Project. https://www.pewresearch.org/global/2014/05/01/public-health-a-major-priority-in-african-nations/

[17] Department of Health Republic of South Africa. (2021). Health Research Priorities (revised) for South Africa 2021-2024 . National Health Research Strategy. https://www.health.gov.za/wp-content/uploads/2022/05/National-Health-Research-Priorities-2021-2024.pdf

[18] Oosthuizen, H. (2013). Legal and Ethical Issues in Stem Cell Research in South Africa. In: Beran, R. (eds) Legal and Forensic Medicine. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-32338-6_80 , see also: Gaobotse G (2018) Stem Cell Research in Africa: Legislation and Challenges. J Regen Med 7:1. doi: 10.4172/2325-9620.1000142

[19] United States Bureau of Citizenship and Immigration Services. (1998). Tunisia: Information on the status of Christian conversions in Tunisia . UNHCR Web Archive. https://webarchive.archive.unhcr.org/20230522142618/https://www.refworld.org/docid/3df0be9a2.html

[20] Gaobotse, G. (2018) Stem Cell Research in Africa: Legislation and Challenges. J Regen Med 7:1. doi: 10.4172/2325-9620.1000142

[21] Kooli, C. Review of assisted reproduction techniques, laws, and regulations in Muslim countries.  Middle East Fertil Soc J   24 , 8 (2020). https://doi.org/10.1186/s43043-019-0011-0 ; Gaobotse, G. (2018) Stem Cell Research in Africa: Legislation and Challenges. J Regen Med 7:1. doi: 10.4172/2325-9620.1000142

[22] Pang M. C. (1999). Protective truthfulness: the Chinese way of safeguarding patients in informed treatment decisions. Journal of medical ethics , 25(3), 247–253. https://doi.org/10.1136/jme.25.3.247

[23] Wang, L., Wang, F., & Zhang, W. (2021). Bioethics in China’s biosecurity law: Forms, effects, and unsettled issues. Journal of law and the biosciences , 8(1).  https://doi.org/10.1093/jlb/lsab019 https://academic.oup.com/jlb/article/8/1/lsab019/6299199

[24] Wang, Y., Xue, Y., & Guo, H. D. (2022). Intervention effects of traditional Chinese medicine on stem cell therapy of myocardial infarction.  Frontiers in pharmacology ,  13 , 1013740. https://doi.org/10.3389/fphar.2022.1013740

[25] Li, X.-T., & Zhao, J. (2012). Chapter 4: An Approach to the Nature of Qi in TCM- Qi and Bioenergy. In Recent Advances in Theories and Practice of Chinese Medicine (p. 79). InTech.

[26] Luo, D., Xu, Z., Wang, Z., & Ran, W. (2021). China's Stem Cell Research and Knowledge Levels of Medical Practitioners and Students.  Stem cells international ,  2021 , 6667743. https://doi.org/10.1155/2021/6667743

[27] Luo, D., Xu, Z., Wang, Z., & Ran, W. (2021). China's Stem Cell Research and Knowledge Levels of Medical Practitioners and Students.  Stem cells international ,  2021 , 6667743. https://doi.org/10.1155/2021/6667743

[28] Zhang, J. Y. (2017). Lost in translation? accountability and governance of Clinical Stem Cell Research in China. Regenerative Medicine , 12 (6), 647–656. https://doi.org/10.2217/rme-2017-0035

[29] Wang, L., Wang, F., & Zhang, W. (2021). Bioethics in China’s biosecurity law: Forms, effects, and unsettled issues. Journal of law and the biosciences , 8(1).  https://doi.org/10.1093/jlb/lsab019 https://academic.oup.com/jlb/article/8/1/lsab019/6299199

[30] Chen, H., Wei, T., Wang, H.  et al.  Association of China’s two-child policy with changes in number of births and birth defects rate, 2008–2017.  BMC Public Health   22 , 434 (2022). https://doi.org/10.1186/s12889-022-12839-0

[31] Azuma, K. Regulatory Landscape of Regenerative Medicine in Japan.  Curr Stem Cell Rep   1 , 118–128 (2015). https://doi.org/10.1007/s40778-015-0012-6

[32] Harris, R. (2005, May 19). Researchers Report Advance in Stem Cell Production . NPR. https://www.npr.org/2005/05/19/4658967/researchers-report-advance-in-stem-cell-production

[33] Park, S. (2012). South Korea steps up stem-cell work.  Nature . https://doi.org/10.1038/nature.2012.10565

[34] Resnik, D. B., Shamoo, A. E., & Krimsky, S. (2006). Fraudulent human embryonic stem cell research in South Korea: lessons learned.  Accountability in research ,  13 (1), 101–109. https://doi.org/10.1080/08989620600634193 .

[35] Alahmad, G., Aljohani, S., & Najjar, M. F. (2020). Ethical challenges regarding the use of stem cells: interviews with researchers from Saudi Arabia. BMC medical ethics, 21(1), 35. https://doi.org/10.1186/s12910-020-00482-6

[36] Association for the Advancement of Blood and Biotherapies.  https://www.aabb.org/regulatory-and-advocacy/regulatory-affairs/regulatory-for-cellular-therapies/international-competent-authorities/saudi-arabia

[37] Alahmad, G., Aljohani, S., & Najjar, M. F. (2020). Ethical challenges regarding the use of stem cells: Interviews with researchers from Saudi Arabia.  BMC medical ethics ,  21 (1), 35. https://doi.org/10.1186/s12910-020-00482-6

[38] Alahmad, G., Aljohani, S., & Najjar, M. F. (2020). Ethical challenges regarding the use of stem cells: Interviews with researchers from Saudi Arabia. BMC medical ethics , 21(1), 35. https://doi.org/10.1186/s12910-020-00482-6

Culturally, autonomy practices follow a relational autonomy approach based on a paternalistic deontological health care model. The adherence to strict international research policies and religious pillars within the regulatory environment is a great foundation for research ethics. However, there is a need to develop locally targeted ethics approaches for research (as called for in Alahmad, G., Aljohani, S., & Najjar, M. F. (2020). Ethical challenges regarding the use of stem cells: interviews with researchers from Saudi Arabia. BMC medical ethics, 21(1), 35. https://doi.org/10.1186/s12910-020-00482-6), this decision-making approach may help advise a research decision model. For more on the clinical cultural autonomy approaches, see: Alabdullah, Y. Y., Alzaid, E., Alsaad, S., Alamri, T., Alolayan, S. W., Bah, S., & Aljoudi, A. S. (2022). Autonomy and paternalism in Shared decision‐making in a Saudi Arabian tertiary hospital: A cross‐sectional study. Developing World Bioethics , 23 (3), 260–268. https://doi.org/10.1111/dewb.12355 ; Bukhari, A. A. (2017). Universal Principles of Bioethics and Patient Rights in Saudi Arabia (Doctoral dissertation, Duquesne University). https://dsc.duq.edu/etd/124; Ladha, S., Nakshawani, S. A., Alzaidy, A., & Tarab, B. (2023, October 26). Islam and Bioethics: What We All Need to Know . Columbia University School of Professional Studies. https://sps.columbia.edu/events/islam-and-bioethics-what-we-all-need-know

[39] Ababneh, M. A., Al-Azzam, S. I., Alzoubi, K., Rababa’h, A., & Al Demour, S. (2021). Understanding and attitudes of the Jordanian public about clinical research ethics.  Research Ethics ,  17 (2), 228-241.  https://doi.org/10.1177/1747016120966779

[40] Ababneh, M. A., Al-Azzam, S. I., Alzoubi, K., Rababa’h, A., & Al Demour, S. (2021). Understanding and attitudes of the Jordanian public about clinical research ethics.  Research Ethics ,  17 (2), 228-241.  https://doi.org/10.1177/1747016120966779

[41] Dajani, R. (2014). Jordan’s stem-cell law can guide the Middle East.  Nature  510, 189. https://doi.org/10.1038/510189a

[42] Dajani, R. (2014). Jordan’s stem-cell law can guide the Middle East.  Nature  510, 189. https://doi.org/10.1038/510189a

[43] The EU’s definition of autonomy relates to the capacity for creating ideas, moral insight, decisions, and actions without constraint, personal responsibility, and informed consent. However, the EU views autonomy as not completely able to protect individuals and depends on other principles, such as dignity, which “expresses the intrinsic worth and fundamental equality of all human beings.” Rendtorff, J.D., Kemp, P. (2019). Four Ethical Principles in European Bioethics and Biolaw: Autonomy, Dignity, Integrity and Vulnerability. In: Valdés, E., Lecaros, J. (eds) Biolaw and Policy in the Twenty-First Century. International Library of Ethics, Law, and the New Medicine, vol 78. Springer, Cham. https://doi.org/10.1007/978-3-030-05903-3_3

[44] Council of Europe. Convention for the protection of Human Rights and Dignity of the Human Being with regard to the Application of Biology and Medicine: Convention on Human Rights and Biomedicine (ETS No. 164) https://www.coe.int/en/web/conventions/full-list?module=treaty-detail&treatynum=164 (forbidding the creation of embryos for research purposes only, and suggests embryos in vitro have protections.); Also see Drabiak-Syed B. K. (2013). New President, New Human Embryonic Stem Cell Research Policy: Comparative International Perspectives and Embryonic Stem Cell Research Laws in France.  Biotechnology Law Report ,  32 (6), 349–356. https://doi.org/10.1089/blr.2013.9865

[45] Rendtorff, J.D., Kemp, P. (2019). Four Ethical Principles in European Bioethics and Biolaw: Autonomy, Dignity, Integrity and Vulnerability. In: Valdés, E., Lecaros, J. (eds) Biolaw and Policy in the Twenty-First Century. International Library of Ethics, Law, and the New Medicine, vol 78. Springer, Cham. https://doi.org/10.1007/978-3-030-05903-3_3

[46] Tomuschat, C., Currie, D. P., Kommers, D. P., & Kerr, R. (Trans.). (1949, May 23). Basic law for the Federal Republic of Germany. https://www.btg-bestellservice.de/pdf/80201000.pdf

[47] Regulation of Stem Cell Research in Germany . Eurostemcell. (2017, April 26). https://www.eurostemcell.org/regulation-stem-cell-research-germany

[48] Regulation of Stem Cell Research in Finland . Eurostemcell. (2017, April 26). https://www.eurostemcell.org/regulation-stem-cell-research-finland

[49] Regulation of Stem Cell Research in Spain . Eurostemcell. (2017, April 26). https://www.eurostemcell.org/regulation-stem-cell-research-spain

[50] Some sources to consider regarding ethics models or regulatory oversights of other cultures not covered:

Kara MA. Applicability of the principle of respect for autonomy: the perspective of Turkey. J Med Ethics. 2007 Nov;33(11):627-30. doi: 10.1136/jme.2006.017400. PMID: 17971462; PMCID: PMC2598110.

Ugarte, O. N., & Acioly, M. A. (2014). The principle of autonomy in Brazil: one needs to discuss it ...  Revista do Colegio Brasileiro de Cirurgioes ,  41 (5), 374–377. https://doi.org/10.1590/0100-69912014005013

Bharadwaj, A., & Glasner, P. E. (2012). Local cells, global science: The rise of embryonic stem cell research in India . Routledge.

For further research on specific European countries regarding ethical and regulatory framework, we recommend this database: Regulation of Stem Cell Research in Europe . Eurostemcell. (2017, April 26). https://www.eurostemcell.org/regulation-stem-cell-research-europe   

[51] Klitzman, R. (2006). Complications of culture in obtaining informed consent. The American Journal of Bioethics, 6(1), 20–21. https://doi.org/10.1080/15265160500394671 see also: Ekmekci, P. E., & Arda, B. (2017). Interculturalism and Informed Consent: Respecting Cultural Differences without Breaching Human Rights.  Cultura (Iasi, Romania) ,  14 (2), 159–172.; For why trust is important in research, see also: Gray, B., Hilder, J., Macdonald, L., Tester, R., Dowell, A., & Stubbe, M. (2017). Are research ethics guidelines culturally competent?  Research Ethics ,  13 (1), 23-41.  https://doi.org/10.1177/1747016116650235

[52] The Qur'an  (M. Khattab, Trans.). (1965). Al-Mu’minun, 23: 12-14. https://quran.com/23

[53] Lenfest, Y. (2017, December 8). Islam and the beginning of human life . Bill of Health. https://blog.petrieflom.law.harvard.edu/2017/12/08/islam-and-the-beginning-of-human-life/

[54] Aksoy, S. (2005). Making regulations and drawing up legislation in Islamic countries under conditions of uncertainty, with special reference to embryonic stem cell research. Journal of Medical Ethics , 31: 399-403.; see also: Mahmoud, Azza. "Islamic Bioethics: National Regulations and Guidelines of Human Stem Cell Research in the Muslim World." Master's thesis, Chapman University, 2022. https://doi.org/10.36837/ chapman.000386

[55] Rashid, R. (2022). When does Ensoulment occur in the Human Foetus. Journal of the British Islamic Medical Association , 12 (4). ISSN 2634 8071. https://www.jbima.com/wp-content/uploads/2023/01/2-Ethics-3_-Ensoulment_Rafaqat.pdf.

[56] Sivaraman, M. & Noor, S. (2017). Ethics of embryonic stem cell research according to Buddhist, Hindu, Catholic, and Islamic religions: perspective from Malaysia. Asian Biomedicine,8(1) 43-52.  https://doi.org/10.5372/1905-7415.0801.260

[57] Jafari, M., Elahi, F., Ozyurt, S. & Wrigley, T. (2007). 4. Religious Perspectives on Embryonic Stem Cell Research. In K. Monroe, R. Miller & J. Tobis (Ed.),  Fundamentals of the Stem Cell Debate: The Scientific, Religious, Ethical, and Political Issues  (pp. 79-94). Berkeley: University of California Press.  https://escholarship.org/content/qt9rj0k7s3/qt9rj0k7s3_noSplash_f9aca2e02c3777c7fb76ea768ba458f0.pdf https://doi.org/10.1525/9780520940994-005

[58] Lecso, P. A. (1991). The Bodhisattva Ideal and Organ Transplantation.  Journal of Religion and Health ,  30 (1), 35–41. http://www.jstor.org/stable/27510629 ; Bodhisattva, S. (n.d.). The Key of Becoming a Bodhisattva . A Guide to the Bodhisattva Way of Life. http://www.buddhism.org/Sutras/2/BodhisattvaWay.htm

[59] There is no explicit religious reference to when life begins or how to conduct research that interacts with the concept of life. However, these are relevant verses pertaining to how the fetus is viewed. (( King James Bible . (1999). Oxford University Press. (original work published 1769))

Jerimiah 1: 5 “Before I formed thee in the belly I knew thee; and before thou camest forth out of the womb I sanctified thee…”

In prophet Jerimiah’s insight, God set him apart as a person known before childbirth, a theme carried within the Psalm of David.

Psalm 139: 13-14 “…Thou hast covered me in my mother's womb. I will praise thee; for I am fearfully and wonderfully made…”

These verses demonstrate David’s respect for God as an entity that would know of all man’s thoughts and doings even before birth.

[60] It should be noted that abortion is not supported as well.

[61] The Vatican. (1987, February 22). Instruction on Respect for Human Life in Its Origin and on the Dignity of Procreation Replies to Certain Questions of the Day . Congregation For the Doctrine of the Faith. https://www.vatican.va/roman_curia/congregations/cfaith/documents/rc_con_cfaith_doc_19870222_respect-for-human-life_en.html

[62] The Vatican. (2000, August 25). Declaration On the Production and the Scientific and Therapeutic Use of Human Embryonic Stem Cells . Pontifical Academy for Life. https://www.vatican.va/roman_curia/pontifical_academies/acdlife/documents/rc_pa_acdlife_doc_20000824_cellule-staminali_en.html ; Ohara, N. (2003). Ethical Consideration of Experimentation Using Living Human Embryos: The Catholic Church’s Position on Human Embryonic Stem Cell Research and Human Cloning. Department of Obstetrics and Gynecology . Retrieved from https://article.imrpress.com/journal/CEOG/30/2-3/pii/2003018/77-81.pdf.

[63] Smith, G. A. (2022, May 23). Like Americans overall, Catholics vary in their abortion views, with regular mass attenders most opposed . Pew Research Center. https://www.pewresearch.org/short-reads/2022/05/23/like-americans-overall-catholics-vary-in-their-abortion-views-with-regular-mass-attenders-most-opposed/

[64] Rosner, F., & Reichman, E. (2002). Embryonic stem cell research in Jewish law. Journal of halacha and contemporary society , (43), 49–68.; Jafari, M., Elahi, F., Ozyurt, S. & Wrigley, T. (2007). 4. Religious Perspectives on Embryonic Stem Cell Research. In K. Monroe, R. Miller & J. Tobis (Ed.),  Fundamentals of the Stem Cell Debate: The Scientific, Religious, Ethical, and Political Issues  (pp. 79-94). Berkeley: University of California Press.  https://escholarship.org/content/qt9rj0k7s3/qt9rj0k7s3_noSplash_f9aca2e02c3777c7fb76ea768ba458f0.pdf https://doi.org/10.1525/9780520940994-005

[65] Schenker J. G. (2008). The beginning of human life: status of embryo. Perspectives in Halakha (Jewish Religious Law).  Journal of assisted reproduction and genetics ,  25 (6), 271–276. https://doi.org/10.1007/s10815-008-9221-6

[66] Ruttenberg, D. (2020, May 5). The Torah of Abortion Justice (annotated source sheet) . Sefaria. https://www.sefaria.org/sheets/234926.7?lang=bi&with=all&lang2=en

[67] Jafari, M., Elahi, F., Ozyurt, S. & Wrigley, T. (2007). 4. Religious Perspectives on Embryonic Stem Cell Research. In K. Monroe, R. Miller & J. Tobis (Ed.),  Fundamentals of the Stem Cell Debate: The Scientific, Religious, Ethical, and Political Issues  (pp. 79-94). Berkeley: University of California Press.  https://escholarship.org/content/qt9rj0k7s3/qt9rj0k7s3_noSplash_f9aca2e02c3777c7fb76ea768ba458f0.pdf https://doi.org/10.1525/9780520940994-005

[68] Gert, B. (2007). Common morality: Deciding what to do . Oxford Univ. Press.

[69] World Medical Association (2013). World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects. JAMA , 310(20), 2191–2194. https://doi.org/10.1001/jama.2013.281053 Declaration of Helsinki – WMA – The World Medical Association .; see also: National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research. (1979).  The Belmont report: Ethical principles and guidelines for the protection of human subjects of research . U.S. Department of Health and Human Services.  https://www.hhs.gov/ohrp/regulations-and-policy/belmont-report/read-the-belmont-report/index.html

[70] Zakarin Safier, L., Gumer, A., Kline, M., Egli, D., & Sauer, M. V. (2018). Compensating human subjects providing oocytes for stem cell research: 9-year experience and outcomes.  Journal of assisted reproduction and genetics ,  35 (7), 1219–1225. https://doi.org/10.1007/s10815-018-1171-z https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6063839/ see also: Riordan, N. H., & Paz Rodríguez, J. (2021). Addressing concerns regarding associated costs, transparency, and integrity of research in recent stem cell trial. Stem Cells Translational Medicine , 10 (12), 1715–1716. https://doi.org/10.1002/sctm.21-0234

[71] Klitzman, R., & Sauer, M. V. (2009). Payment of egg donors in stem cell research in the USA.  Reproductive biomedicine online ,  18 (5), 603–608. https://doi.org/10.1016/s1472-6483(10)60002-8

[72] Krosin, M. T., Klitzman, R., Levin, B., Cheng, J., & Ranney, M. L. (2006). Problems in comprehension of informed consent in rural and peri-urban Mali, West Africa.  Clinical trials (London, England) ,  3 (3), 306–313. https://doi.org/10.1191/1740774506cn150oa

[73] Veatch, Robert M.  Hippocratic, Religious, and Secular Medical Ethics: The Points of Conflict . Georgetown University Press, 2012.

[74] Msoroka, M. S., & Amundsen, D. (2018). One size fits not quite all: Universal research ethics with diversity.  Research Ethics ,  14 (3), 1-17.  https://doi.org/10.1177/1747016117739939

[75] Pirzada, N. (2022). The Expansion of Turkey’s Medical Tourism Industry.  Voices in Bioethics ,  8 . https://doi.org/10.52214/vib.v8i.9894

[76] Stem Cell Tourism: False Hope for Real Money . Harvard Stem Cell Institute (HSCI). (2023). https://hsci.harvard.edu/stem-cell-tourism , See also: Bissassar, M. (2017). Transnational Stem Cell Tourism: An ethical analysis.  Voices in Bioethics ,  3 . https://doi.org/10.7916/vib.v3i.6027

[77] Song, P. (2011) The proliferation of stem cell therapies in post-Mao China: problematizing ethical regulation,  New Genetics and Society , 30:2, 141-153, DOI:  10.1080/14636778.2011.574375

[78] Dajani, R. (2014). Jordan’s stem-cell law can guide the Middle East.  Nature  510, 189. https://doi.org/10.1038/510189a

[79] International Society for Stem Cell Research. (2024). Standards in stem cell research . International Society for Stem Cell Research. https://www.isscr.org/guidelines/5-standards-in-stem-cell-research

[80] Benjamin, R. (2013). People’s science bodies and rights on the Stem Cell Frontier . Stanford University Press.

Mifrah Hayath

SM Candidate Harvard Medical School, MS Biotechnology Johns Hopkins University

Olivia Bowers

MS Bioethics Columbia University (Disclosure: affiliated with Voices in Bioethics)

Article Details

This work is licensed under a Creative Commons Attribution 4.0 International License .

BRIEF RESEARCH REPORT article

How does media contribute to the rise of hate crimes against foreign domestic helpers in hong kong an unfair problem frame and agenda setting provisionally accepted.

• 1 The University of Hong Kong, Hong Kong, SAR China

The final, formatted version of the article will be published soon.

Hate crimes are widespread in Hong Kong society. Foreign domestic helpers working in Hong Kong also experience unfair agenda-setting by the media due to their dual economic and social disadvantages, and the media tries to portray them in a hostile social role. At the same time, the media creates negative social images of minority groups through news coverage, which leads to an increase in social hate crimes against them. This study used WiseSearch, a Chinese newspaper collection and analysis platform, to explore how Hong Kong news media use news themes and content to create a negative image of Hong Kong foreign domestic helpers in order to understand the media origins of hate crimes against Hong Kong foreign domestic helpers. Ultimately, the study found that local news media in Hong Kong are more inclined to cover the legal disputes of foreign domestic helpers in the agenda-setting process. In addition, they are more likely to associate foreign domestic helpers with "fear" rather than "rest assured." The study also found that because of the news value orientation, Hong Kong media tended to treat foreign domestic helpers as outsiders and less sympathetically when writing news stories.

Keywords: hate crimes, Agenda setting, Foreign domestic helpers, Hong Kong, Problem frame approach

Received: 21 Jan 2024; Accepted: 15 May 2024.

Copyright: © 2024 CHEN. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

* Correspondence: Mr. ZHUOLI CHEN, The University of Hong Kong, Pokfulam, Hong Kong, SAR China

People also looked at