Intended for healthcare professionals

CCBYNC Open access
Research

Effectiveness of public health measures in reducing the incidence of covid-19, SARS-CoV-2 transmission, and covid-19 mortality: systematic review and meta-analysis

BMJ 2021; 375 doi: https://doi.org/10.1136/bmj-2021-068302 (Published 18 November 2021) Cite this as: BMJ 2021;375:e068302

Linked Editorial

Public health measures for covid-19

  1. Stella Talic, lecturer in clinical epidemiology and public health12,
  2. Shivangi Shah, honours student1,
  3. Holly Wild, lecturer and honours student13,
  4. Danijela Gasevic, senior lecturer in epidemiology and chronic disease prevention14,
  5. Ashika Maharaj, lecturer quality and safety and cancer epidemiology1,
  6. Zanfina Ademi, associate professor of medical outcomes and health economics12,
  7. Xue Li, assistant professor46,
  8. Wei Xu, research student4,
  9. Ines Mesa-Eguiagaray, statistical geneticist4,
  10. Jasmin Rostron, research student4,
  11. Evropi Theodoratou, professor of cancer epidemiology and global health45,
  12. Xiaomeng Zhang, research student4,
  13. Ashmika Motee, research student4,
  14. Danny Liew, professor of medical outcomes and health economics12,
  15. Dragan Ilic, professor of medical education and public health1
  1. 1School of Public Health and Preventive Medicine, Monash University, Melbourne, 3004 VIC, Australia
  2. 2Monash Outcomes Research and health Economics (MORE) Unit, Monash University, VIC, Australia
  3. 3Torrens University, VIC, Australia
  4. 4Centre for Global Health, The Usher Institute, University of Edinburgh, Edinburgh, UK
  5. 5Cancer Research UK Edinburgh Centre, MRC Institute of Genetics and Molecular Medicine, University of Edinburgh, Edinburgh, UK
  6. 6School of Public Health and The Second Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, China
  1. Correspondence to: S Talic stella.talic{at}monash.edu
  • Accepted 21 October 2021

Abstract

Objective To review the evidence on the effectiveness of public health measures in reducing the incidence of covid-19, SARS-CoV-2 transmission, and covid-19 mortality.

Design Systematic review and meta-analysis.

Data sources Medline, Embase, CINAHL, Biosis, Joanna Briggs, Global Health, and World Health Organization COVID-19 database (preprints).

Eligibility criteria for study selection Observational and interventional studies that assessed the effectiveness of public health measures in reducing the incidence of covid-19, SARS-CoV-2 transmission, and covid-19 mortality.

Main outcome measures The main outcome measure was incidence of covid-19. Secondary outcomes included SARS-CoV-2 transmission and covid-19 mortality.

Data synthesis DerSimonian Laird random effects meta-analysis was performed to investigate the effect of mask wearing, handwashing, and physical distancing measures on incidence of covid-19. Pooled effect estimates with corresponding 95% confidence intervals were computed, and heterogeneity among studies was assessed using Cochran’s Q test and the I2 metrics, with two tailed P values.

Results 72 studies met the inclusion criteria, of which 35 evaluated individual public health measures and 37 assessed multiple public health measures as a “package of interventions.” Eight of 35 studies were included in the meta-analysis, which indicated a reduction in incidence of covid-19 associated with handwashing (relative risk 0.47, 95% confidence interval 0.19 to 1.12, I2=12%), mask wearing (0.47, 0.29 to 0.75, I2=84%), and physical distancing (0.75, 0.59 to 0.95, I2=87%). Owing to heterogeneity of the studies, meta-analysis was not possible for the outcomes of quarantine and isolation, universal lockdowns, and closures of borders, schools, and workplaces. The effects of these interventions were synthesised descriptively.

Conclusions This systematic review and meta-analysis suggests that several personal protective and social measures, including handwashing, mask wearing, and physical distancing are associated with reductions in the incidence covid-19. Public health efforts to implement public health measures should consider community health and sociocultural needs, and future research is needed to better understand the effectiveness of public health measures in the context of covid-19 vaccination.

Systematic review registration PROSPERO CRD42020178692.

Introduction

The impact of SARS-CoV-2 on global public health and economies has been profound.1 As of 14 October 2021, there were 239 007 759 million cases of confirmed covid-19 and 4 871 841 million deaths with covid-19 worldwide.2

A variety of containment and mitigation strategies have been adopted to adequately respond to covid-19, with the intention of deferring major surges of patients in hospitals and protecting the most vulnerable people from infection, including elderly people and those with comorbidities.3 Strategies to achieve these goals are diverse, commonly based on national risk assessments that include estimation of numbers of patients requiring hospital admission and availability of hospital beds and ventilation support.

Globally, vaccination programmes have proved to be safe and effective and save lives.45 Yet most vaccines do not confer 100% protection, and it is not known how vaccines will prevent future transmission of SARS-CoV-2,6 given emerging variants.789 The proportion of the population that must be vaccinated against covid-19 to reach herd immunity depends greatly on current and future variants.10 This vaccination threshold varies according to the country and population’s response, types of vaccines, groups prioritised for vaccination, and viral mutations, among other factors.6 Until herd immunity to covid-19 is reached, regardless of the already proven high vaccination rates,11 public health preventive strategies are likely to remain as first choice measures in disease prevention,12 particularly in places with a low uptake of covid-19 vaccination. Measures such as lockdown (local and national variant), physical distancing, mandatory use of face masks, and hand hygiene have been implemented as primary preventive strategies to curb the covid-19 pandemic.13

Public health (or non-pharmaceutical) interventions have been shown to be beneficial in fighting respiratory infections transmitted through contact, droplets, and aerosols.1415 Given that SARS-CoV-2 is highly transmissible, it is a challenge to determine which measures might be more effective and sustainable for further prevention.

Substantial benefits in reducing mortality were observed in countries with universal lockdowns in place, such as Australia, New Zealand, Singapore, and China. Universal lockdowns are not, however, sustainable, and more tailored interventions need to be considered; the ones that maintain social lives and keep economies functional while protecting high risk individuals.1617 Substantial variation exists in how different countries and governments have applied public health measures,18 and it has proved a challenge for assessing the effectiveness of individual public health measures, particularly in policy decision making.19

Previous systematic reviews on the effectiveness of public health measures to treat covid-19 lacked the inclusion of analytical studies,20 a comprehensive approach to data synthesis (focusing only on one measure),21 a rigorous assessment of effectiveness of public health measures,22 an assessment of the certainty of the evidence,23 and robust methods for comparative analysis.24 To tackle these gaps, we performed a systematic review of the evidence on the effectiveness of both individual and multiple public health measures in reducing the incidence of covid-19, SARS-CoV-2 transmission, and covid-19 mortality. When feasible we also did a critical appraisal of the evidence and meta-analysis.

Methods

This systematic review and meta-analysis were conducted in accordance with PRISMA25 (supplementary material 1, table 1) and with PROSPERO (supplementary material 1, table 2).

Eligibility criteria

Articles that met the population, intervention, comparison, outcome, and study design criteria were eligible for inclusion in this systematic review (supplementary material 1, table 3). Specifically, preventive public health measures that were tested independently were included in the main analysis. Multiple measures, which generally contain a “package of interventions”, were included as supplementary material owing to the inability to report on the individual effectiveness of measures and comparisons on which package led to enhanced outcomes. The public health measures were identified from published World Health Organization sources that reported on the effectiveness of such measures on a range of communicable diseases, mostly respiratory infections, such as influenza.

Given that the scientific community is concerned about the ability of the numerous mathematical models, which are based on assumptions, to predict the course of virus transmission or effectiveness of interventions,26 this review focused only on empirical studies. We excluded case reports and case studies, modelling and simulation studies, studies that provided a graphical summary of measures without clear statistical assessments or outputs, ecological studies that provided a descriptive summary of the measures without assessing linearity or having comparators, non‐empirical studies (eg, commentaries, editorials, government reports), other reviews, articles involving only individuals exposed to other pathogens that can cause respiratory infections, such as severe acute respiratory syndrome or Middle East respiratory syndrome, and articles in a language other than English.

Information sources

We carried out electronic searches of Medline, Embase, CINAHL (Cumulative Index to Nursing and Allied Health Literature, Ebsco), Global Health, Biosis, Joanna Briggs, and the WHO COVID-19 database (for preprints). A clinical epidemiologist (ST) developed the initial search strategy, which was validated by two senior medical librarians (LR and MD) (supplementary material 1, table 4). The updated search strategy was last performed on 7 June 2021. All citations identified from the database searches were uploaded to Covidence, an online software designed for managing systematic reviews,27 for study selection.

Study selection

Authors ST, DG, SS, AM, ET, JR, XL, WX, IME, and XZ independently screened the titles and abstracts and excluded studies that did not match the inclusion criteria. Discrepancies were resolved in discussion with the main author (ST). The same authors retrieved full text articles and determined whether to include or exclude studies on the basis of predetermined selection criteria. Using a pilot tested data extraction form, authors ST, SS, AM, JR, XL, WX, AM, IME, and XZ independently extracted data on study design, intervention, effect measures, outcomes, results, and limitations. ST, SS, AM, and HW verified the extracted data. Table 5 in supplementary material 1 provides the specific criteria used to assess study designs. Given the heterogeneity and diversity in how studies defined public health measures, we took a common approach to summarise evidence of these interventions (supplementary material 1, table 6).

Risk of bias within individual studies

SS, JR, XL, WX, IME, and XZ independently assessed risk of bias for each study, which was cross checked by ST and HW. For non-interventional observational studies, a ROBINS-I (risk of bias in non-randomised studies of interventions) risk of bias tool was used.28 For interventional studies, a revised tool for assessing risk of bias in randomised trials (RoB 2) tool was used.29 Reviewers rated each domain for overall risk of bias as low, moderate, high, or serious/critical.

Data synthesis

The DerSimonian and Laird method was used for random effects meta-analysis, in which the standard error of the study specific estimates was adjusted to incorporate a measure of the extent of variation, or heterogeneity, among the effects observed for public health measures across different studies. It was assumed that the differences between studies are a result of different, yet related, intervention effects being estimated. If fewer than five studies were included in meta-analysis, we applied a recommended modified Hartung-Knapp-Sidik-Jonkman method.30

Statistical analysis

Because of the differences in the effect metrics reported by the included studies, we could only perform quantitative data synthesis for three interventions: handwashing, face mask wearing, and physical distancing. Odds ratios or relative risks with corresponding 95% confidence intervals were reported for the associations between the public health measures and incidence of covid-19. When necessary, we transformed effect metrics derived from different studies to allow pooled analysis. We used the Dersimonian Laird random effects model to estimate pooled effect estimates along with corresponding 95% confidence intervals for each measure. Heterogeneity among individual studies was assessed using the Cochran Q test and the I2 test.31 All statistical analyses were conducted in R (version 4.0.3) and all P values were two tailed, with P=0.05 considered to be significant. For the remaining studies, when meta-analysis was not feasible, we reported the results in a narrative synthesis.

Public and patient involvement

No patients or members of the public were directly involved in this study as no primary data were collected. A member of the public was, however, asked to read the manuscript after submission.

Results

A total of 36 729 studies were initially screened, of which 36 079 were considered irrelevent. After exclusions, 650 studies were eligible for full text review and 72 met the inclusion criteria. Of these studies, 35 assessed individual interventions and were included in the final synthesis of results (fig 1) and 37 assessed multiple interventions as a package and are included in supplementary material 3, tables 2 and 3. The included studies comprised 34 observational studies and one interventional study, eight of which were included in the meta-analysis.

Fig 1
Fig 1

Flow of articles through the review. WHO=World Health Organization

Risk of bias

According to the ROBINS-I tool,28 the risk of bias was rated as low in three studies,323334 moderate in 24 studies,353637383940414243444546474849505152535455565758 and high to serious in seven studies.59606162636465 One important source of serious or critical risk of bias in most of the included studies was major confounding, which was difficult to control for because of the novel nature of the pandemic (ie, natural settings in which multiple interventions might have been enforced at once, different levels of enforcement across regions, and uncaptured individual level interventions such as increased personal hygiene). Variations in testing capacity and coverage, changes to diagnostic criteria, and access to accurate and reliable outcome data on covid-19 incidence and covid-19 mortality, was a source of measurement bias for numerous studies (fig 2). These limitations were particularly prominent early in the pandemic, and in low income environments.4752626365 The randomised controlled trial66 was rated as moderate risk of bias according to the ROB-2 tool. Missing data, losses to follow-up, lack of blinding, and low adherence to intervention all contributed to the reported moderate risk. Tables 1 and 2 in supplementary material 2 summarise the risk of bias assessment for each study assessing individual measures.

Fig 2
Fig 2

Summary of risk of bias across studies assessing individual measures using risk of bias in non-randomised studies of interventions (ROBINS-I) tool

Study characteristics

Studies assessing individual measures

Thirty five studies provided estimates on the effectiveness of an individual public health measures. The studies were conducted in Asia (n=11), the United States (n=9), Europe (n=7), the Middle East (n=3), Africa (n=3), South America (n=1), and Australia (n=1). Thirty four of the studies were observational and one was a randomised controlled trial. The study designs of the observational studies comprised natural experiments (n=11), quasi-experiments (n=3), a prospective cohort (n=1), retrospective cohorts (n=8), case-control (n=2), and cross sectional (n=9). Twenty six studies assessed social measures,3234353738394041424446474852535556575859606163646567 12 studies assessed personal protective measures,3643454950575860636668 three studies assessed travel related measures,545862 and one study assessed environmental measures57 (some interventions overlapped across studies). The most commonly measured outcome was incidence of covid-19 (n=18), followed by SARS-CoV-2 transmission, measured as reproductive number, growth number, or epidemic doubling time (n=13), and covid-19 mortality (n=8). Table 1 in supplementary material 3 provides detailed information on each study.

Effects of interventions

Personal protective measures

Handwashing and covid-19 incidence—Three studies with a total of 292 people infected with SARS-CoV-2 and 10 345 participants were included in the analysis of the effect of handwashing on incidence of covid-19.366063 Overall pooled analysis suggested an estimated 53% non-statistically significant reduction in covid-19 incidence (relative risk 0.47, 95% confidence interval 0.19 to 1.12, I2=12%) (fig 3). A sensitivity analysis without adjustment showed a significant reduction in covid-19 incidence (0.49, 0.33 to 0.72, I2=12%) (fig 4). Risk of bias across the three studies ranged from moderate3660 to serious or critical63 (fig 2).

Fig 3
Fig 3

Meta-analysis of evidence on association between handwashing and incidence of covid-19 using modified Hartung-Knapp-Sidik-Jonkman adjusted random effect model

Fig 4
Fig 4

Meta-analysis of evidence on association between handwashing and incidence of covid-19 using unadjusted random effect model

Mask wearing and covid-19 incidence—Six studies with a total of 2627 people with covid-19 and 389 228 participants were included in the analysis examining the effect of mask wearing on incidence of covid-19 (table 1).364357606366 Overall pooled analysis showed a 53% reduction in covid-19 incidence (0.47, 0.29 to 0.75), although heterogeneity between studies was substantial (I2=84%) (fig 5). Risk of bias across the six studies ranged from moderate36576066 to serious or critical4363 (fig 2).

Table 1

Study characteristics and main results from studies that assessed individual personal protective and environmental measures

View this table:
Fig 5
Fig 5

Meta-analysis of evidence on association between mask wearing and incidence of covid-19 using unadjusted random effect model

Mask wearing and transmission of SARS-CoV-2, covid-19 incidence, and covid-19 mortality—The results of additional studies that assessed mask wearing (not included in the meta-analysis because of substantial differences in the assessed outcomes) indicate a reduction in covid-19 incidence, SARS-CoV-2 transmission, and covid-19 mortality. Specifically, a natural experiment across 200 countries showed 45.7% fewer covid-19 related mortality in countries where mask wearing was mandatory (table 1).49 Another natural experiment study in the US reported a 29% reduction in SARS-CoV-2 transmission (measured as the time varying reproductive number Rt) (risk ratio 0.71, 95% confidence interval 0.58 to 0.75) in states where mask wearing was mandatory.58

A comparative study in the Hong Kong Special Administrative Region reported a statistically significant lower cumulative incidence of covid-19 associated with mask wearing than in selected countries where mask wearing was not mandatory (table 1).68 Similarly, another natural experiment involving 15 US states reported a 2% statistically significant daily decrease in covid-19 transmission (measured as case growth rate) at ≥21 days after mask wearing became mandatory,50 whereas a cross sectional study reported that a 10% increase in self-reported mask wearing was associated with greater odds for control of SARS-CoV-2 transmission (adjusted odds ratio 3.53, 95% confidence interval 2.03 to 6.43).45 The five studies were rated at moderate risk of bias (fig 2).

Environmental measures

Disinfection in household and covid-19 incidence

Only one study, from China, reported the association between disinfection of surfaces and risk of secondary transmission of SARS-CoV-2 within households (table 1).57 The study assessed disinfection retrospectively by asking participants about their “daily use of chlorine or ethanol-based disinfectant in households,” and observed that use of disinfectant was 77% effective at reducing SARS-CoV-2 transmission (odds ratio 0.23, 95% confidence interval 0.07 to 0.84). The study did not collect data on the concentration of the disinfectant used by participants and was rated at moderate risk of bias (fig 2).

Social measures

Physical distancing and covid-19 incidence

Five studies with a total of 2727 people with SARS-CoV-2 and 108 933 participants were included in the analysis that examined the effect of physical distancing on the incidence of covid-19.3753576063 Overall pooled analysis indicated a 25% reduction in incidence of covid-19 (relative risk 0.75, 95% confidence interval 0.59 to 0.95, I2=87%) (fig 6). Heterogeneity among studies was substantial, and risk of bias ranged from moderate37535760 to serious or critical63(fig 2).

Fig 6
Fig 6

Meta-analysis of evidence on association between physical distancing and incidence of covid-19 using unadjusted random effect model

Physical distancing and transmission of SARS-CoV-2 and covid-19 mortality

Studies that assessed physical distancing but were not included in the meta-analysis because of substantial differences in outcomes assessed, generally reported a positive effect of physical distancing (table 2). A natural experiment from the US reported a 12% decrease in SARS-CoV-2 transmission (relative risk 0.88, 95% confidence interval 0.86 to 0.89),40 and a quasi-experimental study from Iran reported a reduction in covid-19 related mortality (β −0.07, 95% confidence interval −0.05 to −0.10; P<0.001).47 Another comparative study in Kenya also reported a reduction in transmission of SARS-CoV-2 after physical distancing was implemented, reporting 62% reduction in overall physical contacts (reproductive number pre-intervention was 2.64 and post-intervention was 0.60 (interquartile range 0.50 to 0.68)).61 These three studies were rated at moderate risk of bias4061 to serious or critical risk of bias47 (fig 2).

Table 2

Study characteristics and main results from studies assessing individual social measures

View this table:

Stay at home or isolation and transmission of SARS-CoV-2

All the studies that assessed stay at home or isolation measures reported reductions in transmission of SARS-CoV-2 (table 2). A retrospective cohort study from the US reported a significant reduction in the odds of having a positive reproductive number (R0) result (odds ratio 0.07, 95% confidence interval 0.01 to 0.37),41 and a natural experiment reported a 51% reduction in time varying reproductive number (Rt) (risk ratio 0.49, 95% confidence interval 0.43 to 0.54).58

A study from the UK reported a 74% reduction in the average daily number of contacts observed for each participant and estimated a decrease in reproductive number: the reproductive number pre-intervention was 3.6 and post-intervention was 0.60 (95% confidence interval 0.37 to 0.89).65 Similarly, an Iranian study projected the reproductive number using serial interval distribution and the number of incidence cases and found a significant decrease: the reproductive number pre-intervention was 2.70 and post-intervention was 1.13 (95% confidence interval 1.03 to 1.25).55 Three of the studies were rated at moderate to serious or critical risk of bias,555865 and one study was rated at low risk of bias41 (fig 2).

Quarantine and incidence and transmission of SARS-CoV-2

Quarantine was assessed in two studies (table 2).3459 A prospective cohort study from Saudi Arabia reported a 4.9% decrease in the incidence of covid-19 at eight weeks after the implementation of quarantine.34 This study was rated at low risk of bias (fig 2). A retrospective cohort study from India reported a 14 times higher risk of SARS-CoV-2 transmission associated with no quarantine compared with strict quarantine (odds ratio 14.44, 95% confidence interval 2.42 to 86.17).59 This study was rated at moderate risk of bias (fig 2).

School closures and covid-19 incidence and covid-19 mortality

Two studies assessed the effectiveness of school closures on transmission of SARS-CoV-2, incidence of covid-19, or covid-19 mortality (table 2).4448 A US population based longitudinal study reported on the effectiveness of state-wide closure of primary and secondary schools and observed a 62% decrease (95% confidence interval −49% to −71%) in incidence of covid-19 and a 58% decrease (−46% to−68%) in covid-19 mortality.48 Conversely, a natural experiment from Japan reported no effect of school closures on incidence of covid-19 (α coefficient 0.08, 95% confidence interval −0.36 to 0.65).44 Both studies were rated at moderate risk of bias (fig 2).

School closures and transmission of SARS-CoV-2

Two natural experiments from the US reported a reduction in transmission (ie, reproductive number); with one study reporting a reduction of 13% (relative risk 0.87, 95% confidence interval 0.86 to 0.89)40 and another reporting a 10% (0.90, 0.86 to 0.93) reduction (table 2).58 A Swedish study reported an association between school closures and a small increase in confirmed SARS-CoV-2 infections in parents (odds ratio 1.17, 95% confidence interval 1.03 to 1.32), but observed that teachers in lower secondary schools were twice as likely to become infected than teachers in upper secondary schools (2.01, 1.52 to 2.67).32 All three studies were rated at moderate risk of bias (fig 2).

Business closures and transmission of SARS-CoV-2

Two natural experiment studies assessed business closures across 50 US states and reported reductions in transmission of SARS-CoV-2 (table 2).4058 One of the studies observed a significant reduction in transmission of 12% (relative risk 0.88, 95% confidence interval 0.86 to 0.89)40 and the other reported a significant 16% (risk ratio 0.84, 0.79 to 0.90) reduction.58 Both studies were rated at moderate risk of bias (fig 2).

Lockdown and incidence of covid-19

A natural experiment involving 202 countries suggested that countries that implemented universal lockdown had fewer new cases of covid-19 than countries that did not (β coefficient −235.8 (standard error −11.04), P<0.01) (table 2).52 An Indian quasi-experimental study reported a 10.8% reduction in incidence of covid-19 post-lockdown,56 whereas a South African retrospective cohort study observed a 14.1% reduction in risk after implementation of universal lockdown (table 2).46 These studies were rated at high risk of bias52 and moderate risk of bias4656 (fig 2).

Lockdown and covid-19 mortality

The three studies that assessed universal lockdown and covid-19 mortality generally reported a decrease in mortality (table 2).353842 A natural experiment study involving 45 US states reported a decrease in covid-19 related mortality of 2.0% (95% confidence interval −3.0% to 0.9%) daily after lockdown had been made mandatory.35 A Brazilian quasi-experimental study reported a 27.4% average difference in covid-19 related mortality rates in the first 25 days of lockdown.42 In addition, a natural experiment study reported about 30% and 60% reductions in covid-19 related mortality post-lockdown in Italy and Spain over four weeks post-intervention, respectively.38 All three studies were rated at moderate risk of bias (fig 2).

Lockdown and transmission of SARS-CoV-2

Four studies assessed universal lockdown and transmission of SARS-CoV-2 during the first few months of the pandemic (table 2). The decrease in reproductive number (R0) ranged from 1.27 in Italy (pre-intervention 2.03, post-intervention 0.76)39 to 2.09 in India (pre-intervention 3.36, post-intervention 1.27),64 and 3.97 in China (pre-intervention 4.95, post-intervention 0.98).33 A natural experiment from the US reported that lockdown was associated with an 11% reduction in transmission of SARS-CoV-2 (relative risk 0.89, 95% confidence interval 0.88 to 0.91).40 All the studies were rated at low risk of bias3339 to moderate risk4064 (fig 2).

Travel related measures

Restricted travel and border closures

Border closure was assessed in one natural experiment study involving nine African countries (table 3).62 Overall, the countries recorded an increase in the incidence of covid-19 after border closure. These studies concluded that the implementation of border closures within African countries had minimal effect on the incidence of covid-19. The study had important limitations and was rated at serious or critical risk of bias. In the US, a natural experiment study reported that restrictions on travel between states contributed about 11% to a reduction in SARS-CoV-2 transmission (table 3).36 The study was rated at moderate risk of bias (fig 2).

Table 3

Study characteristics and main results from studies that assessed individual travel measures

View this table:

Entry and exit screening (virus or symptom screening)

One retrospective cohort study assessed screening of symptoms, which involved testing 65 000 people for fever (table 3).54 The study found that screening for fever lacked sensitivity (ranging from 18% to 24%) in detecting people with SARS-CoV-2 infection. This translated to 86% of the population with SARS-CoV-2 remaining undetected when screening for fever. The study was rated at moderate risk of bias (fig 2).

Multiple public health measures

Study characteristics

Overall, 37 studies provided estimates on the effectiveness of multiple public health measures, assessed as a collective group. Studies were mostly conducted in Asia (n=15), the US (n=11), Europe (n=6), Africa (n=4), and South America (n=1). All the studies were observational. The most commonly measured outcome was transmission of disease (ie, measured as reproductive number, growth number, or epidemic doubling time) (n=23), followed by covid-19 incidence (n=19) and covid-19 mortality (n=8). This review attempted to assess the overall effectiveness of the public health intervention packages by reporting the percentage difference in outcome before and after implementation of measures or between regions or countries studied. Eleven of the 37 included studies noted a difference of between 26% and 50% in transmission of SARS-CoV-2 and incidence of covid-19,7071727374757677787980 nine noted a difference of between 51% and 75% in SARS-CoV-2 transmission, covid-19 incidence, and covid-19 mortality,818283848586878889 and 14 noted a difference of more than 75% in transmission of SARS-CoV-2, covid-19 incidence and covid-19 mortality.79808990919293949596979899100 For the remaining studies, the overall effectiveness was not assessed owing to a lack of comparators (see supplementary material 3, table 3). Two studies that assessed universal lockdown and physical distancing reported a decrease of between 0% and 25% in SARS-CoV-2 transmission and covid-19 incidence.79101 Studies that included school and workplace closures,919596 isolation or stay at home measures,8094 or a combination of both798993979899 reported decreases of more than 75% in SARS-CoV-2 transmission. Supplementary material 3, table 2 provides detailed information on each study.

Discussion

Worldwide, government and public health organisations are mitigating the spread of SARS-CoV-2 by implementing various public health measures. This systematic review identified a statistically significant reduction in the incidence of covid-19 through the implementation of mask wearing and physical distancing. Handwashing interventions also indicated a substantial reduction in covid-19 incidence, albeit not statistically significant in the adjusted model. As the random effects model tends to underestimate confidence intervals when a meta-analysis includes a small number of individual studies (<5), the adjusted model for handwashing showed a statistically non-significant association in reducing the incidence of covid-19 compared with the unadjusted model.

Overall effectiveness of these interventions was affected by clinical heterogeneity and methodological limitations, such as confounding and measurement bias. It was not possible to evaluate the impact of type of face maks (eg, surgical, fabric, N95 respirators) and compliance and frequency of wearing masks owing to a lack of data. Similarly, it was not feasible to assess the differences in effect that different recommendations for physical distancing (ie, 1.5 m, 2m, or 3 m) have as preventive strategies.

The effectiveness of measures such as universal lockdowns and closures of businesses and schools for the containment of covid-19 have largely been effective, but depended on early implementation when incidence rates of covid-19 were still low.425258 Only Japan reported no decrease in covid-19 incidence after school closures,44 and other studies found that different public health measures were sometimes implemented simultaneously or soon after one another, thus the results should be interpreted with caution.324656

Isolation or stay at home was an effective measure in reducing the transmission of SARS-CoV-2, but the included studies used results for mobility to assess stay at home or isolation and therefore could have been limited by potential flaws in publicly available phone data,4158102 and variations in the enforcement of public health measures in different states or regions were not assessed.5558102 Quarantine was found to be as effective in reducing the incidence of covid-19 and transmission of SARS-CoV-2, yet variation in testing and case detection in low income environments was substantial.599698 Another study reported that quarantine was effective in reducing the transmission of SARS-CoV-2 in a cohort with a low prevalence of the virus, yet it is unknown if the same effect would be observed with higher prevalence.34

It was not possible to draw conclusions about the effectiveness of restricted travel and full border closures because the number of empirical studies was insufficient. Single studies identified that border closure in Africa had a minimal effect in reducing SARS-CoV-2 transmission, but the study was assessed as being at high risk of bias.62 Screening for fever was also identified to be ineffective, with only 24% of positive cases being captured by screening.54

Comparison with other studies

Previous literature reviews have identified mask wearing as an effective measure for the containment of SARS-CoV-2103; the caveat being that more high level evidence is required to provide unequivocal support for the effectiveness of the universal use of face masks.104105 Additional empirical evidence from a recent randomised controlled trial (originally published as a preprint) indicates that mask wearing achieved a 9.3% reduction in seroprevalence of symptomatic SARS-CoV-2 infection and an 11.9% reduction in the prevalence of covid-19-like symptoms.106 Another systematic review showed stronger effectiveness with the use of N95, or similar, respirators than disposable surgical masks,107 and a study evaluating the protection offered by 18 different types of fabric masks found substantial heterogeneity in protection, with the most effective mask being multilayered and tight fitting.108 However, transmission of SARS-CoV-2 largely arises in hospital settings in which full personal protective measures are in place, which suggests that when viral load is at its highest, even the best performing face masks might not provide adequate protection.51 Additionally, most studies that assessed mask wearing were prone to important confounding bias, which might have altered the conclusions drawn from this review (ie, effect estimates might have been underestimated or overestimated or can be related to other measures that were in place at the time the studies were conducted). Thus, the extent of such limitations on the conclusions drawn remain unknown.

A 2020 rapid review concluded that quarantine is largely effective in reducing the incidence of covid-19 and covid-19 mortality. However, uncertainty over the magnitude of such an effect still remains,109 with enhanced management of quality quarantine facilities for improved effective control of the epidemics urgently needed.110 In addition, findings on the application of school and workplace closures are still inconclusive. Policy makers should be aware of the ambiguous evidence when considering school closures, as other potentially less disruptive physical distancing interventions might be more appropriate.21 Numerous findings from studies on the efficacy of school closures showed that the risk of transmission within the educational environment often strongly depends on the incidence of covid-19 in the community, and that school closures are most successfully associated with control of SARS-CoV-2 transmission when other mitigation strategies are in place in the community.111112113114115116117 School closures have been reported to be disruptive to students globally and are likely to impair children’s social, psychological, and educational development118119 and to result in loss of income and productivity in adults who cannot work because of childcare responsibilities.120

Speculation remains as how best to implement physical distancing measures.121 Studies that assess physical distancing measures might interchangeably study physical distancing with lockdown35525664 and other measures and thus direct associations are difficult to assess.

Empirical evidence from restricted travel and full border closures is also limited, as it is almost impossible to study these strategies as single measures. Current evidence from a recent narrative literature review suggested that control of movement, along with mandated quarantine, travel restrictions, and restricting nationals from entering areas of high infection, are effective measures, but only with good compliance.122 A narrative literature review of travel bans, partial lockdowns, and quarantine also suggested effectiveness of these measures,123 and another rapid review further supported travel restrictions and cross border restrictions to stop the spread of SARS-CoV-2.124 It was impossible to make such observations in the current review because of limited evidence. A German review, however, suggested that entry, exit, and symptom screening measures to prevent transmission of SARS-CoV-2 are not effective at detecting a meaningful proportion of cases,125 and another review using real world data from multiple countries found that border closures had minimal impact on the control of covid-19.126

Although universal lockdowns have shown a protective effect in lowering the incidence of covid-19, SARS-CoV-2 transmission, and covid-19 mortality, these measures are also disruptive to the psychosocial and mental health of children and adolescents,127 global economies,128 and societies.129 Partial lockdowns could be an alternative, as the associated effectiveness can be high,125 especially when implemented early in an outbreak,85 and such measures would be less disruptive to the general population.

It is important to also consider numerous sociopolitical and socioeconomic factors that have been shown to increase SARS-CoV-2 infection130131 and covid-19 mortality.132 Immigration status,82 economic status,81101 and poverty and rurality98 can influence individual and community compliance with public health measures. Poverty can impact the ability of communities to physically distance,133 especially in crowded living environments,134135 as well as reduce access to personal protective measures.134135 A recent study highlights that “a one size fits all” approach to public health measures might not be effective at reducing the spread of SARS-CoV-2 in vulnerable communities136 and could exacerbate social and economic inequalities.135137 As such, a more nuanced and community specific approach might be required. Even though screening is highly recommended by WHO138 because a proportion of patients with covid-19 can be asymptomatic,138 screening for symptoms might miss a larger proportion of the population with covid-19. Hence, temperature screening technologies might need to be reconsidered and evaluated for cost effectiveness, given such measures are largely depended on symptomatic fever cases.

Strengths and limitations of this review

The main strength of this systematic review was the use of a comprehensive search strategy to identify and select studies for review and thereby minimise selection bias. A clinical epidemiologist developed the search strategy, which was validated by two senior medical librarians. This review followed a comprehensive appraisal process that is recommended by the Cochrane Collaboration31 to assess the effectiveness of public health measures, with specifically validated tools used to independently and individually assess the risk of bias in each study by study design.

This review has some limitations. Firstly, high quality evidence on SARS CoV-2 and the effectiveness of public health measures is still limited, with most studies having different underlying target variables. Secondly, information provided in this review is based on current evidence, so will be modified as additional data become available, especially from more prospective and randomised studies. Also, we excluded studies that did not provide certainty over the effect measure, which might have introduced selection bias and limited the interpretation of effectiveness. Thirdly, numerous studies measured interventions only once and others multiple times over short time frames (days v month, or no timeframe). Additionally, the meta-analytical portion of this study was limited by significant heterogeneity observed across studies, which could neither be explored nor explained by subgroup analyses or meta-regression. Finally, we quantitatively assessed only publications that reported individual measures; studies that assessed multiple measures simultaneously were narratively analysed with a broader level of effectiveness (see supplementary material 3, table 3). Also, we excluded studies in languages other than English.

Methodological limitations of studies included in the review

Several studies failed to define and assess for potential confounders, which made it difficult for our review to draw a one directional or causal conclusion. This problem was mainly because we were unable to study only one intervention, given that many countries implemented several public health measures simultaneously; thus it is a challenge to disentangle the impact of individual interventions (ie, physical distancing when other interventions could be contributing to the effect). Additionally, studies measured different primary outcomes and in varied ways, which limited the ability to statistically analyse other measures and compare effectiveness.

Further pragmatic randomised controlled trials and natural experiment studies are needed to better inform the evidence and guide the future implementation of public health measures. Given that most measures depend on a population’s adherence and compliance, it is important to understand and consider how these might be affected by factors. A lack of data in the assessed studies meant it was not possible to understand or determine the level of compliance and adherence to any of the measures.

Conclusions and policy implications

Current evidence from quantitative analyses indicates a benefit associated with handwashing, mask wearing, and physical distancing in reducing the incidence of covid-19. The narrative results of this review indicate an effectiveness of both individual or packages of public health measures on the transmission of SARS-CoV-2 and incidence of covid-19. Some of the public health measures seem to be more stringent than others and have a greater impact on economies and the health of populations. When implementing public health measures, it is important to consider specific health and sociocultural needs of the communities and to weigh the potential negative effects of the public health measures against the positive effects for general populations. Further research is needed to assess the effectiveness of public health measures after adequate vaccination coverage has been achieved. It is likely that further control of the covid-19 pandemic depends not only on high vaccination coverage and its effectiveness but also on ongoing adherence to effective and sustainable public health measures.

What is already known on this topic

  • Public health measures have been identified as a preventive strategy for influenza pandemics

  • The effectiveness of such interventions in reducing the transmission of SARS-CoV-2 is unknown

What this study adds

  • The findings of this review suggest that personal and social measures, including handwashing, mask wearing, and physical distancing are effective at reducing the incidence of covid-19

  • More stringent measures, such as lockdowns and closures of borders, schools, and workplaces need to be carefully assessed by weighing the potential negative effects of these measures on general populations

  • Further research is needed to assess the effectiveness of public health measures after adequate vaccination coverage

Ethics statements

Ethical approval

Not required.

Data availability statement

No additional data available.

Acknowledgments

We thank medical subject librarians Lorena Romero (LR) and Marshall Dozier (MD) for their expert advice and assistance with the study search strategy.

Footnotes

  • Contributors: ST, DG, DI, DL, and ZA conceived and designed the study. ST, DG, SS, AM, HW, WX, JR, ET, AM, XL, XZ, and IME collected and screened the data. ST, DG, and DI acquired, analysed, or interpreted the data. ST, HW, and SS drafted the manuscript. All authors critically revised the manuscript for important intellectual content.. XL and ST did the statistical analysis. NA obtained funding. LR and MD provided administrative, technical, or material support. ST and DI supervised the study. ST and DI had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. ST is the guarantor. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.

  • Funding: No funding was available for this research. ET is supported by a Cancer Research UK Career Development Fellowship (grant No C31250/A22804). XZ is supported by The Darwin Trust of Edinburgh.

  • Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/disclosure-of-interest/and declare: ET is supported by a Cancer Research UK Career Development Fellowship and XZ is supported by The Darwin Trust of Edinburgh; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; and no other relationships or activities that could appear to have influenced the submitted work.

  • The lead author (ST) affirms that the manuscript is an honest, accurate, and transparent account of the study reported; no important aspects of the study have been omitted. Dissemination to participants and related patient and public communities: It is anticipated to disseminate the results of this research to wider community via press release and social media platforms.

  • Provenance and peer review: Not commissioned; externally peer reviewed.

http://creativecommons.org/licenses/by-nc/4.0/

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

References