Abstract
Background The Pneumonia Severity Index (PSI) and the CURB-65 score assess disease severity in patients with community-acquired pneumonia (CAP). We compared the clinical performance of both prognostic scores according to clinical outcomes and admission rates.
Methods A nationwide retrospective cohort study was conducted using claims data from adult CAP patients presenting to the emergency department (ED) in 2018 and 2019. Dutch hospitals were divided into three categories: “CURB-65 hospitals” (n=25), “PSI hospitals” (n=19) and hospitals using both (“no-consensus hospitals”, n=15). Main outcomes were hospital admission rates, intensive care unit admissions, length of hospital stay, delayed admissions, readmissions and all-cause 30-day mortality. Multilevel logistic and Poisson regression analysis were used to adjust for potential confounders.
Findings Of 50 984 included CAP patients, 21 157 were treated in CURB-65 hospitals, 17 279 in PSI hospitals and 12 548 in no-consensus hospitals. The 30-day mortality was significantly lower in CURB-65 hospitals versus PSI hospitals (8.6% and 9.7%, adjusted odds ratio (aOR) 0.89, 95% CI: 0.83–0.96, p=0.003). Other clinical outcomes were similar between CURB-65 hospitals and PSI hospitals. No-consensus hospitals had higher admission rates compared to the CURB-65 and PSI hospitals combined (78.4% and 81.5%, aOR 0.78, 95% CI: 0.62–0.99).
Interpretation In this study, using the CURB-65 in CAP patients at the ED is associated with similar and possibly even better clinical outcomes compared to using the PSI. After confirmation in prospective studies, the CURB-65 may be recommended over the use of the PSI since it is associated with lower 30-day mortality and is more user-friendly.
Abstract
In this study, using the CURB-65 in CAP patients at the ED in the Netherlands is associated with similar and possibly even better clinical outcomes compared to the PSI. After further confirmation, the CURB-65 may be recommended over the use of the PSI. https://bit.ly/41UktKX
Introduction
Community-acquired pneumonia (CAP) is one of the most common infectious diseases, causing substantial clinical and economic burden worldwide [1–3]. To guide physicians in estimating disease severity in CAP patients, prediction tools such as the Pneumonia Severity Index (PSI) and the CURB-65 score have been developed [4, 5]. The PSI is a prognostic score based on 20 variables including demographics, comorbidities and clinical variables. The CURB-65 consists of five elements: confusion, uraemia, respiratory rate, blood pressure and age ≥65 years [4, 5]. The PSI was developed to identify patients with a low 30-day mortality risk and is considered a safe way to determine need for hospitalisation [4, 6–9]. While the CURB-65 was developed to stratify patients into mortality risk groups with separate management strategies, no studies investigated the effect of the CURB-65 on admission rates [5]. The prognostic performance of both scores is considered to be similar by most studies, although some found the PSI to perform better in predicting mortality and identifying low-risk patients [10–13].
International guidelines have yet to reach consensus on which prediction tool to prefer for clinical practice and how to use it. The British Thoracic Society and National Institute for Health and Care Excellence (NICE) guidelines recommend using the CURB-65 for severity assessment, and for deciding on treatment strategy and treatment location in adults with CAP [14, 15]. Contrarily, the American Thoracic Society and Infectious Diseases Society of America (IDSA) recommends the PSI as a decision aid only to determine treatment location, while stating that there is a paucity of evidence regarding the effectiveness and safety of the CURB-65 [16].
The Dutch CAP guideline recommends using a prognostic tool to assess disease severity and to decide on treatment strategy [17]. Importantly, the guideline does not prefer either the PSI or the CURB-65. This gave us the unique opportunity to compare the clinical performance of both prognostic scores on a large scale, by comparing clinical outcomes of CAP patients treated at hospitals that use the PSI to those treated at hospitals that use the CURB-65. The aim of this study is to evaluate the PSI and the CURB-65 in clinical practice, by comparing clinical outcomes and admission rates of CAP patients presenting to the emergency department (ED).
Methods
Setting and data sources
We conducted a retrospective study using claims data provided by the centre for information of Dutch health insurers, Vektis, to the National Healthcare Institute (NHCI) of the Netherlands. Vektis collects data from all health insurers which include, among other, insurance claims of medical specialist care and medication claims of pharmacies.
Claims data
In the Dutch healthcare system, hospital care is financed through diagnosis treatment combinations (DTCs). All patients receiving hospital care are assigned a DTC according to their diagnosis. This DTC covers all hospital activities and interventions that a patient with that diagnosis might receive, and is specific for both diagnosis and medical specialty (e.g. DTC pneumonia, department of internal medicine).
Patients
All patients aged ≥18 years with a pneumonia DTC started between 1 January 2018 and 31 December 2019, with primary involvement of internal medicine or pulmonology, were selected. Of those, only patients with an ED visit, including the performance of chest imaging (radiograph or computed tomography), at the start of the DTC were included, as that would be the setting in which the PSI and CURB-65 are used. If a patient had more than one episode of pneumonia, only the first was included. To restrict to community-acquired cases, patients who had been discharged from a hospital up to 30 days prior to the DTC registration were excluded.
Variables and outcomes
Demographic information including age, sex and comorbidities was available in the claims database. A neighbourhood socioeconomic status (SES) from 2016 was available through Netherlands Bureau for Economic Policy Analysis (CPB). This is measured using status scores indicating the relative status of a neighbourhood. It is derived from education, income and position on the labour market and was divided into quartiles for analysis (a higher quartile indicating a higher SES) [18]. Comorbidities were identified through additional DTC registrations. DTCs started within 5 years prior to the pneumonia DTC were included to include current comorbidities. These DTCs were classified into larger categories, based on the categories in the PSI as defined by Fine et al. [4]: neoplastic disease, liver disease, congestive heart failure, cerebrovascular disease and chronic renal disease. In addition, we assessed the presence of cardiovascular disease, pulmonary disease, diabetes mellitus, immunosuppression, neurologic disease and HIV infection. Smoking status or body mass index was not available.
By assessing DTCs only, information on relevant diseases treated by a general practitioner could be missed. Therefore, chronic use of immunosuppressants and diabetes medication in the year prior to the pneumonia was identified through medication claims of pharmacies. Patients were considered immunocompromised if they used at least one immunosuppressant in combination with a DTC suggesting impaired immunity. The diabetes group comprised patients either using diabetes medication, having a diabetes DTC or both.
The main outcomes of this study were hospital admission rate, readmissions, intensive care unit (ICU) admissions, delayed admissions, all-cause 30-day mortality and length of hospital stay. Readmission was defined as a discharge from the hospital followed by admission within 7 days. Delayed admission was defined as initial outpatient treatment, followed by admission to the hospital within 7 days. More detailed definitions, an overview of DTCs per category and included medication can be found in the supplementary appendix.
Hospitals
The NHCI conducted a survey in March 2020 among all pulmonologists and internal medicine specialists in the Netherlands asking which prognostic score for CAP was used in their hospital during 2018–2019 and how this prognostic score was used [19]. Response from at least one physician from both departments was required. If this was not the case, a physician on call from the missing specialty was contacted by phone. We received a response from 93% of all hospitals.
After collecting all survey results and answers received by phone, hospitals were divided into “PSI hospitals”, “CURB-65 hospitals” and “no-consensus hospitals”. This last category consists of hospitals using both prognostic scores within the same department or different prognostic scores between internal medicine and pulmonology departments. As claims data were hospital specific, we were able to specify which prognostic score was used for severity assessment in the selected patients.
Statistical analysis
For each outcome, patients from “PSI hospitals” were compared to patients from “CURB-65 hospitals” and “no-consensus hospitals”. In addition, patients from “no-consensus hospitals” were compared to patients from PSI and CURB-65 hospitals as one group (“consensus”), to examine differences between hospitals that use one prognostic score and hospitals that use both.
Crude and adjusted odds ratios (ORs) with 95% confidence intervals (95% CI) were calculated for all outcomes. Multilevel logistic regression analyses (iterative generalised least squares (IGLS), 2nd order predictive quasi-likelihood (PQL), random intercept at hospital level) were performed for dichotomous outcomes. A multilevel Poisson linear regression analysis (IGLS, 2nd order PQL, random intercept at hospital level) was performed for length of stay, with a correction for overdispersion. We corrected for possible confounders: age, sex, neighbourhood SES, comorbidities, specialism (internal medicine or pulmonology) and type of hospital (general or teaching hospital). Furthermore, a variable was added reflecting the number of days that patients were alive in the investigated period to account for time-dependent risk, except for the analysis of 30-day mortality. For 30-day mortality, three multilevel regression subgroup analyses were performed based on predefined age categories, admission (yes/no) and ICU admission (yes/no). p-values of <0.05 were considered statistically significant. SAS Enterprise Guide 7.1 was used for the descriptive statistics. MLwiN 3.05 was used for the multilevel analyses.
Results
Hospitals
A total of 59 059 patients from 73 Dutch hospitals met the eligibility criteria (figure 1). Patients from the eight university hospitals were excluded as they were all either CURB-65 hospitals or no-consensus hospitals, making a comparison to university PSI hospitals impossible. One hospital used neither PSI or CURB-65 and was therefore excluded. Five other hospitals were excluded, as we were unable to determine which prognostic score was used. Finally, 257 patients from whom the zip code and therefore the SES was not available were excluded. A total of 50 984 patients from 59 hospitals were included in the final analysis (figure 1).
Flowchart of hospital and patient selection. PSI: Pneumonia Severity Index; SES: socioeconomic status.
Questionnaire
The respondents to the questionnaire estimated that in 79% of patients a prognostic score was used. 83% of the respondents used the prognostic score to decide on antibiotic treatment, 49% used it to determine the need for hospitalisation, and 28% used it to decide whether a patient should be treated at an ICU [19].
Baseline characteristics
24 062 patients were female (47%), and the median age was 72 years (table 1). The most prevalent comorbidities across all 50 984 patients were cardiovascular disease (10 525, 21%), pulmonary disease (9971, 20%), neoplastic disease (9253, 18%) and diabetes mellitus (9276, 18%). The patients from PSI hospitals were more likely to have a SES in the lowest quartile compared to the other patients. Overall, 36 343 out of 50 984 patients were managed by the pulmonology department (71%). The majority of the consensus hospitals were classified as teaching hospitals; 61% of CURB-65 hospitals and 62% of PSI hospitals. In contrast, most of the no-consensus hospitals were general hospitals (60%) (table 1).
Baseline patient characteristics
Outcomes
Overall, 40 368 out of 50 984 (79.2%) patients were admitted to the hospital directly after their ED visit and 4341 out of 50 984 (8.5%) were admitted to the ICU. The all-cause 30-day mortality was 9.1% (4628 out of 50 984, table 1). Out of 10 616 patients primarily treated as outpatients, 832 (7.8%) were admitted within 7 days after presentation at the ED. Of all 41 200 combined admissions and delayed admissions, 977 patients (2.4%) were readmitted within 7 days after discharge.
CURB-65 versus PSI
The 30-day mortality was significantly lower in the CURB-65 hospitals compared to the PSI hospitals; 8.6% versus 9.7% (table 2). This association was statistically significant with an aOR of 0.89 (95% CI: 0.83–0.96, p=0.003). There was a trend towards lower admission rates in the CURB-65 hospitals than in the PSI hospitals; 77.2% versus 79.9% (aOR 0.81, 95% CI: 0.64–1.02). ICU admission rates, readmissions and delayed admissions were similar between the CURB-65 and PSI hospitals (table 2). Lastly, for all admitted patients, the median length of stay was 5 days (IQR 4–8) for the CURB-65 hospitals and 6 days (IQR 4–8) for the PSI hospitals, with an adjusted relative risk (aRR) of 0.99 (95% CI: 0.94–1.04).
Multilevel analysis of outcomes of patients with community-acquired pneumonia using CURB-65 versus Pneumonia Severity Index (PSI)
Consensus versus no-consensus
Admission rates were significantly lower in consensus hospitals compared to no-consensus hospitals: 78.4% versus 81.5% (aOR 0.78, 95% CI: 0.62–0.99, table 3). Other outcomes were all similar, including the median length of stay (aRR 1.03, 95% CI: 0.98–1.08).
Multilevel analysis of outcomes of patients with community-acquired pneumonia using either CURB-65 or Pneumonia Severity Index (PSI) versus no-consensus hospitals
Subgroup analyses
In the analysis by age, most subgroups showed a decrease in mortality when treated in a CURB-65 hospital versus a PSI hospital (figure 2). There was a stronger association in the 50–64 years age group (aOR 0.71, 95% CI: 0.55–0.92). In the age group 18 to 49, no reliable estimate could be made due to the low mortality rate. The subgroup analysis by admission showed similar associations for admitted patients (aOR 0.90, 95% CI: 0.83–0.97) and for those treated as outpatients (aOR 0.91, 95% CI: 0.72–1.15). Finally, for both patients with and without ICU admission, the CURB-65 was associated with a decrease in 30-day mortality (figure 2).
Subgroup analysis for the primary outcome 30-day mortality in patients with community-acquired pneumonia: CURB-65 versus Pneumonia Severity Index (PSI). ICU: intensive care unit.
Discussion
We found that using the CURB-65 as a severity assessment tool among ED CAP patients was associated with a lower risk of 30-day mortality compared to the PSI. In addition, the admission rates were higher among hospitals that did not consistently use only one of the prognostic scores in the management of CAP. Based on this large multicentre study, using the CURB-65 in CAP patients at the ED leads to similar and possibly even better clinical outcomes compared to the PSI.
The overall 30-day mortality rate in this study was 9.1%, slightly higher than mortality rates in previous Dutch studies of 6.7% and 6.8% [20, 21]. This difference could be explained by the fact that the patients in these studies were on average younger with less comorbidities. Our mortality rate was in line with the PSI study by Fine et al. [4], and the CURB-65 study by Lim et al. [5], showing mortality rates of 10.2% and 9%, respectively.
It is a striking finding that the CURB-65 is associated with a lower 30-day mortality compared to the PSI. There are several potential explanations for this observation. Several studies showed that the CURB-65 classifies more patients as “severe CAP” (score 3–5) compared to the PSI (score of 5) [12, 20, 22]. This finding, combined with the fact that the Dutch CAP guideline recommends using the CURB-65 or the PSI for deciding upon antibiotic treatment, could potentially lead to different empirical treatment in CURB-65 versus PSI hospitals [17]. We do not have data on antibiotic use; however, since 83% of respondents use the prognostic score to support their choice of antibiotics, it seems realistic to think that there has been a difference in treatment between the CURB-65 and PSI hospitals [19]. In severe CAP, it is advised to start with a 2nd or 3rd generation cephalosporin intravenously, whereas in mild and moderate–severe CAP, narrow-spectrum β-lactam antibiotics (including oral regimens) are recommended. Thus, categorising more patients as “severe CAP” when using CURB-65 would likely have resulted in more patients being treated with cephalosporins, which might explain the lower mortality rate. The fact that the difference was especially visible in the age category of 50 to 65 years would also support the hypothesis, since the PSI tends to underestimate disease severity for younger patients [23]. Another explanation could be that since more patients are judged as severe CAP when using CURB-65, more patients received intensive monitoring and associated supportive care measures resulting in better outcome. One could hypothesise that patients in CURB-65 hospitals were less ill, especially since they were less often admitted. However, the analysis of only admitted patients showed similar results. In another analysis, we included the academic hospitals which were seven CURB-65 hospitals and one no-consensus hospital. One would expect that academic hospitals treat patients with a higher risk of adverse outcomes. However, this also did not change our results (see supplementary appendix). Of course, it is possible that residual confounding accounts for the observed difference. Notably, the PSI hospitals treated patients with relatively lower SES. Although analyses were corrected for SES as a confounder, low SES is for instance associated with smoking, something that we could not take into account [24]. Finally, it should be noted that international guidelines differ in whether prognostic scores should be used to guide antibiotic treatment or (ICU) admission policy. Such differences should be kept in mind when interpreting mortality outcomes in an international setting.
Another interesting finding is that we found lower admission rates in consensus hospitals compared to no-consensus hospitals, without an increase in adverse outcomes. This suggests that not consistently using a prognostic score within one hospital could lead to suboptimal admission policy. It has been shown that using the PSI is associated with lower admission rates without compromising safety outcomes [6–9]. To our knowledge, there are currently no data regarding the CURB-65 and its association with hospitalisation rates. In this study, the admission rates in the CURB-65 hospitals were similar to the PSI hospitals, with a trend for even lower admission rates (77.2% versus 79.9%; OR 0.81, 95% CI: 0.64–1.02). This could be explained by the fact that the CURB-65 does not only classify more patients in the high-risk categories, but also classifies more patients into the lower risk categories when compared to the PSI [12, 17, 20].
The major strength of this study is the large study population, representing the whole population across a Western country. Another strength is the inclusion of patients from 2 years, limiting seasonal effects. Moreover, only patients from 2018 and 2019 were included, i.e. before the COVID-19 epidemic. However, using claims data inevitably leads to important limitations. First, we did not have access to information at the individual level such as received treatment, diagnostic results, resuscitation codes and the actual patient's risk score. However, it is not to be expected that the patient populations would be significantly different across the three hospital categories. Second, we have limited information on guideline adherence other than the questionnaire results. Our hypotheses to explain the observed differences are based on the assumption that the patient's risk classification leads to a specific treatment. Although it is known that guideline adherence is suboptimal, we do not have a reason to assume that guideline adherence would be different between the PSI and the CURB-65 hospitals [20]. Furthermore, the questionnaire results suggest that the majority of respondents do use the prognostic score to decide on treatment. Third, we have limited information about differences between hospitals, such as differences in local treatment guidelines, different hospital facilities or differences in expertise of treating physicians. However, the difference in quality of care between Dutch hospitals is generally considered to be minimal. Furthermore, we corrected all analysis for type of hospital and all hospital categories were evenly distributed across the different geographical regions. Finally, we tried to categorise each hospital as accurately as possible. Although unlikely, it cannot be fully ruled out that a hospital was misclassified.
To our knowledge, this is the largest study comparing clinical outcomes when using the PSI or CURB-65 in clinical practice of adult CAP patients. Previous studies have mainly focused on the predictive abilities, with most studies finding similar results for the PSI and the CURB-65, while our study focused on clinical outcomes [10, 11]. Although claims data inevitably has its limitations, we believe that this study shows that consistently using the PSI or CURB-65 in one hospital is associated with lower admission rates without increasing adverse outcomes. Furthermore, using the CURB-65 in CAP patients at the ED is associated with similar and possibly even better clinical outcomes compared to the PSI. Since the CURB-65 is also more user-friendly than the PSI as it only contains five variables, we conclude that after confirmation of our findings in well-conducted prospective studies, the CURB-65 may be recommended over the use of the PSI.
Supplementary material
Supplementary Material
Please note: supplementary material is not edited by the Editorial Office, and is uploaded as it has been supplied by the author.
Supplementary material 00051-2023.SUPPLEMENT
Footnotes
Provenance: Submitted article, peer reviewed.
Ethics: According to Dutch legislation, there was no need for approval by a medical ethics committee since we used pseudo-anonymised claims data.
Data statement: Due to privacy legislation, underlying claims data cannot be made available.
Author contributions: Conceptualisation: A.G. Kaal, L. op de Hoek, D.T. Hochheimer, C. Brouwers, W.J. Wiersinga, D. Snijders, K.L. Rensing and C. van Nieuwkoop. Data curation: A.G. Kaal, L. op de Hoek, D.T. Hochheimer and C.E. van Dijk. Software: A.G. Kaal, L. op de Hoek, D.T. Hochheimer and C.E. van Dijk. Formal analysis: A.G. Kaal, L. op de Hoek, D.T. Hochheimer and C.E. van Dijk. Supervision: E.W. Steyerberg and C. van Nieuwkoop. Validation: all authors. Investigation: A.G. Kaal, L. op de Hoek, D.T. Hochheimer and C.E. van Dijk. Visualisation: A.G. Kaal and L. op de Hoek. Methodology: A.G. Kaal, L. op de Hoek, D.T. Hochheimer, C. Brouwers, K.L. Rensing, C.E. van Dijk and C. van Nieuwkoop. Writing (original draft): A.G. Kaal and L. op de Hoek. Writing (review and editing): all authors. All authors had access to the data, and the underlying data were verified by A.G. Kaal, L. op de Hoek, D.T. Hochheimer and C.E. van Dijk.
Conflict of interest: W.J. Wiersinga reports two grants, the Vidi grant from the Dutch Research Council, and a grant on COVID-19 from The Netherlands Organisation for Health Research and Development. Furthermore, his host institution received ad hoc consultancy fees from GSK (DSMB), Pfizer, AstraZeneca and Sobi. All other authors report no conflicts of interest.
- Received January 25, 2023.
- Accepted February 6, 2023.
- Copyright ©The authors 2023
This version is distributed under the terms of the Creative Commons Attribution Non-Commercial Licence 4.0. For commercial reproduction rights and permissions contact permissions{at}ersnet.org
References
