The inter-rater reliability of the Pulmonary Embolism Severity Index

 

 Buy Article Permissions and Reprints

Summary

The Pulmonary Embolism Severity Index (PESI) is a validated clinical prognostic model for patients with acute pulmonary embolism (PE). Our goal was to assess the PESI’s inter-rater reliability in patients diagnosed with PE. We prospectively identified consecutive patients diagnosed with PE in the emergency department of a Swiss teaching hospital. For all patients, resident and attending physician raters independently collected the 11 PESI variables. The raters then calculated the PESI total point score and classified patients into one of five PESI risk classes (I-V) and as low (risk classes I/II) versus higher-risk (risk classes III-V). We examined the inter-rater reliability for each of the 11 PESI variables, the PESI total point score, assignment to each of the five PESI risk classes, and classification of patients as low versus higher-risk using kappa (κ) and intra-class correlation coefficients (ICC). Among 48 consecutive patients with an objective diagnosis of PE, reliability coefficients between resident and attending physician raters were > 0.60 for 10 of the 11 variables comprising the PESI. The inter-rater reliability for the PESI total point score (ICC: 0.89, 95% CI: 0.81–0.94), PESI risk class assignment (κ: 0.81, 95% CI: 0.66–0.94), and the classification of patients as low versus higher-risk (κ: 0.92, 95% CI: 0.72–0.98) was near perfect. Our results demonstrate the high reproducibility of the PESI, supporting the use of the PESI for risk stratification of patients with PE.

• References

• 1 Aujesky D, Obrosky DS, Stone RA. et al. Derivation and validation of a prognostic model for pulmonary embolism. Am J Respir Crit Care Med 2005; 172: 1041-1046.
  CrossrefPubMedGoogle Scholar
• 2 Aujesky D, Roy PM, Le Manach CP. et al. Validation of a model to predict adverse outcomes in patients with pulmonary embolism. Eur Heart J 2006; 27: 476-481.
  CrossrefPubMedGoogle Scholar
• 3 Jimenez D, Yusen RD, Otero R. et al. Prognostic models for selecting patients with acute pulmonary embolism for initial outpatient therapy. Chest 2007; 132: 24-30.
  CrossrefPubMedGoogle Scholar
• 4 Donze J, Le Gal G, Fine MJ. et al. Prospective validation of the Pulmonary Embolism Severity Index. A clinical prognostic model for pulmonary embolism. Thromb Haemost 2008; 100: 943-948.
  Article in Thieme ConnectPubMedGoogle Scholar
• 5 Wicki J, Perrier A, Perneger TV, Bounameaux H, Junod AF. Predicting adverse outcome in patients with acute pulmonary embolism: a risk score. Thromb Haemost 2000; 84: 548-552.
  Article in Thieme ConnectPubMedGoogle Scholar
• 6 Sanchez O, Trinquart L, Caille V. et al. Prognostic Factors for Pulmonary Embolism: The PREP Study, A Prospective Multicenter Cohort Study. Am J Respir Crit Care Med 2009; 181: 168-173.
  PubMedGoogle Scholar
• 7 Laupacis A, Sekar N, Stiell IG. Clinical prediction rules. A review and suggested modifications of methodological standards. J Am Med Assoc 1997; 277: 488-494.
  CrossrefPubMedGoogle Scholar
• 8 Cohen JA. Coefficient of agreement for nominal scales. Ed Psychol Meas 1960; 20: 37-46.
  CrossrefPubMedGoogle Scholar
• 9 Cohen J. Weighted kappa: nominal scale agreement with provision for scaled disagreement or partial credit. Psychol Bull 1968; 70: 213-220.
  CrossrefPubMedGoogle Scholar
• 10 Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull 1979; 86: 420-428.
  CrossrefPubMedGoogle Scholar
• 11 Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977; 33: 159-174.
  CrossrefPubMedGoogle Scholar
• 12 Sim J, Wright CC. The kappa statistic in reliability studies: use, interpretation, and sample size requirements. Phys Ther 2005; 85: 257-268.
  CrossrefPubMedGoogle Scholar
• 13 Chan CM, Woods C, Shorr AF. The validation and reproducibility of the Pulmonary Embolism Severity Index. J Thromb Haemost 2010; 8: 1509-1514.
  CrossrefPubMedGoogle Scholar
• 14 Nordenholz KE, Naviaux NW, Stegelmeier K. et al. Pulmonary embolism risk assessment screening tools: the interrater reliability of their criteria. Am J Emerg Med 2007; 25: 285-290.
  CrossrefPubMedGoogle Scholar
• 15 Wolf SJ, McCubbin TR, Feldhaus KM. et al. Prospective validation of Wells Criteria in the evaluation of patients with suspected pulmonary embolism. Ann Emerg Med 2004; 44: 503-510.
  CrossrefPubMedGoogle Scholar
• 16 Runyon MS, Webb WB, Jones AE. et al. Comparison of the unstructured clinician estimate of pretest probability for pulmonary embolism to the Canadian score and the Charlotte rule: a prospective observational study. Acad Emerg Med 2005; 12: 587-593.
  CrossrefPubMedGoogle Scholar
• 17 Jiménez D, Aujesky D, Moores L. et al. Simplification of the Pulmonary Embolism Severity Index for prognostication in patients with acute symptomatic pulmonary embolism. Arch Intern Med 2010; 170: 1383-1389.
  CrossrefPubMedGoogle Scholar