Review
Measurement error is often neglected in medical literature: a systematic review

https://doi.org/10.1016/j.jclinepi.2018.02.023Get rights and content

Abstract

Objectives

In medical research, covariates (e.g., exposure and confounder variables) are often measured with error. While it is well accepted that this introduces bias and imprecision in exposure-outcome relations, it is unclear to what extent such issues are currently considered in research practice. The objective was to study common practices regarding covariate measurement error via a systematic review of general medicine and epidemiology literature.

Study Design and Setting

Original research published in 2016 in 12 high impact journals was full-text searched for phrases relating to measurement error. Reporting of measurement error and methods to investigate or correct for it were quantified and characterized.

Results

Two hundred and forty-seven (44%) of the 565 original research publications reported on the presence of measurement error. 83% of these 247 did so with respect to the exposure and/or confounder variables. Only 18 publications (7% of 247) used methods to investigate or correct for measurement error.

Conclusions

Consequently, it is difficult for readers to judge the robustness of presented results to the existence of measurement error in the majority of publications in high impact journals. Our systematic review highlights the need for increased awareness about the possible impact of covariate measurement error. Additionally, guidance on the use of measurement error correction methods is necessary.

Introduction

Measurement error is one of many key challenges to make valid inferences in biomedical research [1]. Errors in measurements can arise due to inaccuracy or imprecision of measurement instruments, data coding errors, self-reporting, or single measurements of variable longitudinal processes, such as biomarkers. With the increased use of data not originally intended for research, such as routine care data, “claims” databases, and other sources of “big data”, it is conceivable that measurement error is becoming increasingly prevalent in this field [2].

It is generally well accepted that measurement error and classification error (hereinafter collectively referred to as measurement error) in either the dependent variable (hereinafter outcome) or independent explanatory variables (hereinafter covariates; e.g., exposure and confounder variables) can introduce bias and imprecision to estimates of covariate–outcome relations. Among others, several textbooks [3], [4], [5], [6], methodological reviews [7], [8], and a tool-kit [9] have demonstrated how to examine, quantify, and correct for measurement error in a variety of settings encountered in epidemiology. Most of this work has been focused on measurement error in covariates given its conceived greater impact on studied relations than measurement error in the outcome [4]. Despite these resources, it is suspected that the attention it receives in applied medical and epidemiological studies is insufficient [10], [11].

Over a decade ago, a review of 57 randomly selected publications from three high-ranking epidemiology journals reported that 61% of the reviewed publications recognized the potential influence of measurement error, but only 28% made a qualitative assessment of its impact on their results, and only one quantified its potential impact on results [12]. In light of the increasing prevalence of measurement error in medical and epidemiological research and increasing availability of methods and software to account for measurement error, a new and more comprehensive investigation into current practice is necessary.

We conducted a systematic review to quantify the extent to which (possible) measurement error in covariates is addressed in recent medical and epidemiologic research published in high-impact journals. To guide the understanding of the results of the review, we briefly introduce key concepts in the field of measurement error.

Section snippets

Measurement error

Many variables of interest in medical research are subject to measurement error. Instead of an error-free and unobserved true value of a variable, researchers have to deal with an imperfectly measured observed value. For the remainder of this section, we consider the erroneous measurement and perfect measurement of a single underlying entity as different variables. Examples of variables prone to measurement error include the long-term average level of a variable biological process (such as

Methods

We performed a systematic review of original research published in 2016 in high-impact medical and epidemiological journals. Our aims were to (i) quantify and characterize the reporting of measurement error in a main exposure and/or confounder variables and their possible impact on study results and (ii) quantify and characterize the use of available methods for investigating or correcting for measurement error in the exposure and/or confounder variables.

Using the Thomson Reuters [20] InCites

Results

Fig. 1 depicts the number of included articles at each step of the review process. Of the 1178 articles found in PubMed, 565 (337 from Epidemiology journals and 228 from General & Internal Medicine Journals) were judged as original research satisfying our inclusion criteria. Of these, 247 (44%) directly addressed measurement error in some form. Characteristics of these included studies are found in Table 1. Eighteen of these publications (3% of the 565) investigated the possible impact of, or

Discussion

This review provides an overview of the attention given to measurement error in recent epidemiological and medical literature. We found that a high proportion (44%) reported on the (possible) presence of measurement error in one or more recorded variables. About 70% of these addressed measurement error in a qualitative manner only in the discussion section. In contrast, few publications (7%) used some form of measurement error analysis to investigate or correct the exposure–outcome relation for

References (48)

  • M.W. Knuiman et al.

    Adjustment for regression dilution in epidemiological regression analyses

    Ann Epidemiol

    (1998)
  • Z. Obermeyer et al.

    Predicting the future—big data, machine learning, and clinical medicine

    N Engl J Med

    (2016)
  • W.A. Fuller

    Measurement error models

    (1987)
  • P. Gustafson

    Measurement error and misclassification in statistics and epidemiology: impacts and bayesian adjustments

    (2004)
  • R.J. Carroll et al.

    Measurement error in nonlinear models: a modern perspective

    (2006)
  • J. Buonaccorsi

    Measurement error: models, methods and applications

    (2010)
  • L.A. Stefanski

    Measurement error models

    J Am Stat Assoc

    (2000)
  • A. Guolo

    Robust techniques for measurement error correction: a review

    Stat Methods Med Res

    (2008)
  • R. Keogh et al.

    A toolkit for measurement error correction, with a focus on nutritional epidemiology

    Stat Med

    (2014)
  • J.S. Buzas et al.

    Measurement error

  • M. Blackwell et al.

    A unified approach to measurement error and missing data: overview and applications

    Sociol Methods Res

    (2015)
  • A.M. Jurek et al.

    Exposure-measurement error is frequently ignored when interpreting epidemiologic study results

    Eur J Epidemiol

    (2006)
  • T.B. Brakenhoff et al.

    Random measurement error: why worry? An example of cardiovascular risk factors

    PLoS One

    (2018)
  • B. Rosner et al.

    Correction of logistic regression relative risk estimates and confidence intervals for systematic within-person measurement error

    Stat Med

    (1989)
  • J. Cook et al.

    Simulation-extrapolation estimation in parametric measurement error models

    J Am Stat Assoc

    (1994)
  • H. Küchenhoff et al.

    A general method for dealing with misclassification in regression: the misclassification SIMEX

    Biometrics

    (2006)
  • A. Skrondal et al.

    Generalized latent variable modeling: multilevel, longitudinal, and structural equation models

    (2004)
  • R.B. Kline

    Principles and practice of structural equation modeling

    (2015)
  • Thomson Reuters

    InCites journal citation reports

    (2016)
  • K.A. Guertin et al.

    Coffee consumption and incidence of lung cancer in the NIH-AARP Diet and Health Study

    Int J Epidemiol

    (2016)
  • M. Song et al.

    Long-term status and change of body fat distribution, and risk of colorectal cancer: a prospective cohort study

    Int J Epidemiol

    (2016)
  • S.B. Pedersen et al.

    Nonfasting mild-to-moderate hypertriglyceridemia and risk of acute pancreatitis

    JAMA Intern Med

    (2016)
  • Cited by (83)

    • Sensitivity analysis for random measurement error using regression calibration and simulation-extrapolation

      2021, Global Epidemiology
      Citation Excerpt :

      Measurement error is common in biomedical research but often ignored [1,2].

    View all citing articles on Scopus

    Funding: This work was supported by the Netherlands Organization for Scientific Research (NWO-Vidi project 917.16.430 granted to R.H.H.G).

    Conflicts of interest: None.

    View full text