Introduction

Faster respiratory pathogen detection and antibiotic resistance identification are important in critical care due to the severity of illness, significant prior antibiotic exposure and infection control implications. Many respiratory viral and atypical bacterial panel assays are commercially available and offer rapid molecular detection for both high- and low-throughput laboratory and near-patient test settings [1, 2]. However, rapid, automated commercial molecular assays for the typical respiratory bacteria targeted by most broad-spectrum empirical antibiotic regimens are significantly fewer in number and lack important antibiotic resistance gene targets [3, 4]. Therefore, antibiotic de-escalation and pathogen-targeted antimicrobial therapy for severe pneumonia identified in the intensive care unit (ICU) still relies on the results of slow, traditional microbiological culture methods.

We recently developed in-house fast real-time quantitative PCR assays for typical respiratory bacteria [5] and demonstrated that a comprehensive molecular testing approach for sputum in community-acquired pneumonia (CAP) significantly improved microbiological diagnosis and had the potential to enable early pathogen-directed antibiotic therapy [6]. However, this approach requires a number of individual multiplex reactions for each specimen and does not incorporate antibiotic resistance gene detection. The Unyvero P55 Pneumonia Cartridge (Curetis AG) is a commercially available rapid molecular assay which can detect 21 respiratory pathogen species and genera alongside 17 antibiotic resistance genes [7]. The test uses an automated sample-to-answer format with minimal hands-on time and takes approximately 5 h, with the potential to be used as a point-of-care test. Targets comprise two Gram-positive bacteria, 15 Gram-negative bacteria and four agents of atypical pneumonia: Legionella pneumophila, Mycoplasma pneumoniae, Chlamydophila pneumoniae and Pneumocystis jirovecii.

The aim of the study was to compare the performance of the Unyvero P55 Pneumonia Cartridge to standard culture-based methods and our in-house bacterial molecular diagnostic assays on bronchoalveolar lavage fluids (BALs) from ICU patients. We also intended to estimate the potential impact of rapid molecular assays on antibiotic prescribing in this setting.

Methods

Specimens and patients

Seventy-four BAL fluids from 74 consecutive individual patients admitted to the Royal Infirmary of Edinburgh ICU between 1 January 2013 and 30 September 2015 were tested using routine microbiological culture and molecular methods. Residual specimens were diluted approximately 1/10 in UTM viral transport medium (Copan) and stored at − 80 °C. Microbiology test results were obtained from the laboratory database for each BAL sample and combined with anonymised clinical data and information on antimicrobial prescription. Suspected ventilator-associated pneumonia (VAP) was defined as the presence of clinical and radiological signs of pneumonia in a patient mechanically ventilated for ≥ 48 h. Suspected hospital-acquired pneumonia (HAP) was defined as the presence of clinical and radiological signs of pneumonia in a patient hospitalised for ≥ 48 h. Suspected community-acquired pneumonia (CAP) was defined as the presence of clinical and radiological signs of pneumonia in a patient hospitalised for < 48 h. Study approval was granted by NHS Lothian Caldicott Guardian and by NHS Lothian SAHSC Bioresource (SR211), with ethical approval to test anonymised, excess patient samples (10/S1402/33).

Prospective testing

Standard microbiological culture, biochemical and/or MALDI-TOF (Bruker) identification and automated antibiotic sensitivity testing (VITEK2, bioMérieux) were carried out according to national standard protocols [8]. All specimens were also tested by viral and atypical agent real-time PCRs adapted from previously published assays or developed in-house [9,10,11,12,13,14]. CMV real-time PCR was performed routinely on all specimens using the Sentosa SA CMV Quantitative PCR Test (Vela Diagnostics). Total nucleic acid was extracted from specimens using the automated nucliSENS easyMAG (bioMerieux) system with off-board lysis, input volume of 200 μl and elution volume of 100 μl. The same extract was used in multiplex real-time PCR assays in a panel comprising: (1) influenza A, influenza B and respiratory syncytial virus; (2) parainfluenza viruses 1, 2 and 3; (3) adenovirus and Mycoplasma pneumoniae; (4) rhinovirus and human metapneumovirus; (5) Legionella species and L. pneumophila; (6) CMV and (7) HSV-1 and 2. Real-time PCR for Pneumocystis jirovecii was carried out following specific clinical request.

Retrospective testing

Thawed BAL specimens were tested using both commercial and in-house bacterial respiratory panels after a period of storage at − 80 °C ranging from 8 to 41 months. For the Unyvero P55 pneumonia assay (Curetis AG), specimens were processed according to the manufacturer’s instructions and two specimens were processed simultaneously. A 180 μl specimen was added to the sample tube, placed into the lysator for 30 min and then placed into the test cartridge and loaded into the analyser. Targets for the Unyvero P55 assay are as follows: Staphylococcus aureus, Streptococcus pneumoniae, Citrobacter freundii, Escherichia coli, Enterobacter cloacae complex, Enterobacter aerogenes, Proteus spp., Klebsiella pneumoniae, K. oxytoca, K. variicola, Serratia marcescens, Morganella morganii, Moraxella catarrhalis, Pseudomonas aeruginosa, Acinetobacter baumannii complex, Stenotrophomonas maltophilia, Haemophilus influenzae, L. pneumophila, M. pneumoniae, C. pneumoniae, P. jirovecii and resistance genes: ermB, mecA, mecC, tem, shv, ctx-M, kpc, imp, ndm, oxa-23, oxa-24/40, oxa-48, oxa-58, vim, sul1, gyrA83 and gyrA87. For in-house respiratory bacteria multiplex real-time PCR, 200 μl specimen was treated with lysozyme at 37 °C followed by proteinase K at 56 °C for 1 h each. Total nucleic acid was then extracted using the automated nucliSENS easyMAG (BioMérieux) instrument with an off-board lysis protocol, including the addition of phocine herpesvirus as an internal extraction control. Fast quantitative real-time PCR was performed using two multiplex assays developed for research use: (1) S. pneumoniae, H. influenzae, M. catarrhalis, S. aureus and (2) E. coli, K. pneumoniae, P. aeruginosa, A. baumannii [5].

Statistical analysis

Two-by-two contingency tables of categorical variables were analysed by either Fisher’s exact test or chi-square with Yates’ correction (GraphPad, CA, USA) to assess the effect of antibiotic exposure on test positivity. For sensitivity and specificity calculations, routine culture methodology was considered to be the gold standard.

Results

Patient characteristics

BAL fluids were available for retrospective testing from 74 patients admitted to the intensive care unit during the study period. Patients were admitted to the unit for a wide variety of reasons, reflecting the hospital’s function as a major tertiary care centre (Table 1). However, BAL was performed predominantly as part of the diagnostic workup for pneumonia, particularly ventilator-associated pneumonia (Table 1).

Table 1 Characteristics of included patients (n = 74) with BAL fluid tested in the study
Full size table

Prospective BAL fluid testing

BAL fluids available for the study were equally distributed over the 33 month study period with 29/74 (39.2%) in 2013, 24/74 (32.4%) in 2014 and 21/74 (28.4%) in 2015. Prospective microbiological testing by standard culture and viral and atypical PCR screening identified an organism in 70/74 (94.6%) specimens, with more than one organism reported in 46/74 (62.2%) specimens (Table 2). Bacteria were isolated by routine culture in 48/74 (65%) specimens, with Gram-negative bacteria in 42/74 (56.8%) specimens. Of the 17 bacterial species cultured, 11/17 (64.7%) were present in the Unyvero P55 panel and 6/17 (35.3%) were present in the in-house PCR panel. Yeasts were the most frequently isolated group of fungal organisms, being present in 30/74 (40.5%) specimens, whilst viruses were detected in 21/74 (28.4%) specimens.

Table 2 Detection of microorganisms in BAL fluids by routine microbiological culture and viral PCR (n = 74)
Full size table

Retrospective molecular testing

Compared to the bacterial culture gold standard, overall sensitivity/specificity was 56.9%/58.5% for the Unyvero P55 panel and 63.2%/54.8% for the in-house PCR panel (Table 3). Cultured bacteria were not detected by the Unyvero P55 and in-house PCR panel in 25 (33.8%) and 21 (28.4%) specimens respectively (Table 3). For Unyvero P55, 19/25 (76.0%) missed detections were clinically significant bacteria present in the panel. For in-house PCR, 7/21 (33.3%) missed detections were clinically significant bacteria present in the panel and 9/21(42.9%) were clinically significant bacteria not present in the panel (Table S1). Single isolates of six rare respiratory or clinically non-significant organisms were not detected by either molecular assay as targets were not present in either panel (Table S1). Additional organisms were detected by Unyvero P55 and in-house PCR panels in 12 (16.2%) and 12 (16.2%) specimens respectively (Table 3 and Table S2). The Unyvero P55 panel made 17 additional bacterial detections; 8/10 (80%) of these with corresponding panel targets were also positive by in-house PCR. The in-house PCR panel made 19 additional bacterial detections; 8/19 (42.1%) of these were also positive by Unyvero P55 (Table S2).

Table 3 Comparison of molecular panel tests to bacterial culture gold standard for bacterial identification in BAL specimens
Full size table

Compared to the bacterial culture gold standard, in-panel sensitivity/specificity was 63.5%/58.5% for the Unyvero P55 panel and 83.7%/54.8% for the in-house PCR panel (Table 3).

Antibiotic susceptibility

Unyvero P55 antibiotic susceptibility prediction by the presence or absence of resistance genes was compared to the gold standard automated culture-based phenotypic testing, giving a sensitivity of 18.8%, specificity of 94.9%, NPV of 74.2% and a non-calculable PPV (Table 4). In particular, phenotypic amoxicillin, third-generation cephalosporin and macrolide/lincosamide resistance were poorly detected by the molecular test, with 8/25 (32%), 0/8 (0%) and 0/8 (0%) resistant isolates detected respectively. Furthermore, of the five instances of fluoroquinolone resistance detection by molecular assay, four (80%) were false positive. There were an additional 12 instances of resistance gene detection where the corresponding antibiotic susceptibility phenotype was unknown. This was due to either the antibiotic not being present in the phenotypic battery tested or no growth from the specimen: these were sul1 (6), mecA (3), ermB (2) and tem (1).

Table 4 Comparison of Unyvero P55 panel antibiotic resistance gene detection to the phenotypic antibiotic susceptibility testing gold standard in BAL specimens
Full size table

Antibiotic prescribing

Data on antibiotic prescription in ITU were available for 58 (78.4%) patients in the study. On the day of the bronchoalveolar lavage procedure, 37/58 (63.8%) patients were receiving at least one antibiotic. The frequency of bacterial detection by routine culture was significantly higher in patients not receiving antibiotics (18/22, 81.8%) than in those receiving antibiotics (18/36, 50.0%), (difference 31.8% (95%CI 6.2 to 50.8% (p = 0.025)). Detection frequency was not affected by antibiotic prescription for either the Unyvero P55 panel (12/35 (34.3%) vs. 11/22 (50.0%) respectively, p = 0.368) or the in-house PCR panel (17/36 (47.2%) vs.10/22 (45.5%) respectively, p = 0.896). The turnaround time from electronic test request to final electronic reporting of all prospective microbiological investigations on the BAL fluid sample was a median of 3 days [range 2–16 days]. However, once-daily microbiology consultation ward rounds occurred 7 days per week in the unit, with discussion of interim results before final reporting. Antimicrobial prescription and any changes in light of BAL fluid results were documented in the patient’s medical records.

Antimicrobials were changed on the basis of the BAL fluid routine microbiological testing results in 35 (60.3%) cases (Table S3); 19 (54.3%) changes were de-escalations in number and/or spectrum of agent(s) and 14 (40.0%) changes were escalations. In particular, 28 of the 35 antimicrobial changes are related to antibiotic changes. These decisions were re-examined in light of retrospective molecular test panel results to estimate the predicted benefits and risks of acting on molecular test identification results before culture results were available. In 15 (53.6%) and 17 (60.7%) cases respectively, acting on Unyvero P55 or in-house PCR panel identification results may have brought potential benefits as true-positive or true-negative results would have been available sooner (Table 5). However, there was a risk of a false-negative result being acted upon in six (21.4%) cases with either molecular test. Risks and benefits were unclear in seven (25.0%) and five (17.9%) cases with Unyvero P55 and in-house PCR panels respectively. This was due to additional detections of unclear significance retrospectively or true-positive and false-negative detections for the same specimen.

Table 5 Estimated potential benefits and risks of acting on results from molecular panel assays before culture results available (n = 28)
Full size table

Discussion

Microbiological investigation of lower respiratory tract infection in patients in the critical care setting would benefit from new methodologies with increased sensitivity and turnaround time. The aetiology of pneumonia is broad and few commercial respiratory panel assays are available with a sufficiently wide range of targets, particularly for the Gram-positive and Gram-negative bacteria seen frequently in ventilator and hospital-associated pneumonia [3, 4, 15,16,17]. In addition to this, rapid antibiotic resistance detection should enable more appropriate antibiotic selection to improve patient management, earlier antibiotic de-escalation to aid stewardship and earlier infection control interventions [18]. With this in mind, we tested the Unyvero P55 Pneumonia Cartridge assay retrospectively on a microbiologically well-characterised BAL specimen set to try to establish the potential benefits of using this approach in our ICU setting versus our in-house bacterial real-time multiplex PCR assays [5, 6].

As expected, the majority of our patients were undergoing BAL for suspected VAP, HAP or CAP and we isolated a wide range of Gram-positive and Gram-negative bacterial species from two-thirds of cases. The target coverage of cultured bacteria by the two molecular panels was higher at 89.9% for Unyvero P55 compared to 72.9% for the in-house PCR panel, and the few isolates cultured that were not in the Unyvero P55 panel were either rarely isolated pathogens or of doubtful significance. Fungal and viral pathogens absent from the Unyvero P55 panel were frequently identified by culture and routine PCR; however, the clinical significance of many of these was also unclear in this retrospective study. As expected, herpesvirus reactivation in this critical care cohort was common with HSV-1 and CMV, the most frequently detected viruses in BAL specimens. However, 3/14 patients with HSV-1 positive BAL specimens and 2/6 patients with CMV positive BAL specimens did receive specific antiviral treatment, indicating that significance was interpreted in the individual clinical context including concurrent viraemia.

We found that overall bacterial and target-specific sensitivity of the Unyvero P55 on BAL specimens was low at 56.9 and 63.5% respectively. In-house PCR assay performed better with overall and target-specific sensitivity of 63.2 and 83.7% respectively. For the Unyvero P55 test, we found that missed detections were predominantly bacterial targets which were present in the panel, whereas, for the in-house PCR assay, missed detections were mainly bacterial targets not present in the panel. Specificity of both tests was similar at 58.5% for the Unyvero P55 panel and 54.8% for the in-house PCR panel, with additional organisms detected by both tests in 16.2% specimens. Sensitivity of Unyvero P55 has been assessed in two previous publications; our sensitivity estimate is more in line with that of Drick et al. [7] (66%) who also used BAL specimens than that of Ozongwu et al. [19] (89%) who predominantly used sputa and endotracheal specimens; therefore, specimen type may potentially affect the performance of the assay. Where additional detections were made by Unyvero P55 for targets present in our in-house molecular assay, we could corroborate 80% of them, suggesting that these were real additional positives, as also found in a previous study using molecular comparators [19]. An explanation is that the majority of patients in our study received antibiotics on the day of the BAL procedure and this was associated with reduced culture detection frequency, whereas molecular detection frequency was unaffected. Therefore, a bacterial culture may not be the appropriate “gold standard” for comparison. Many additional in-house PCR detections were negative by Unyvero P55, and this may reflect the higher sensitivity of in-house PCR compared to Unyvero P55, as seen in culture-positive specimens.

The sensitivity of the Unyvero P55 panel for antibiotic resistance gene detection was low at 18.8%, with particular problems detecting aminopenicillin, third-generation cephalosporin and macrolide/lincosamide resistance. In agreement with our study, Ozongwu et al. also found variable ability of the Unyvero P55 assay to detect antimicrobial resistance, despite excluding the results of tem, shv, ermB and sul1 gene detection. The existence of multiple genetic variants and the fact that mechanisms of resistance are also present in non-pathogenic respiratory tract microflora mean that accurate and relevant resistance gene detection direct from clinical specimens is not straightforward.

Diagnosis of VAP is in part reliant on a microbiological quantitative culture threshold of > 10 colony forming units (CFU)/ml [15] and we and others have demonstrated that molecular quantification of bacteria in BALs from VAP patients can be useful diagnostically [16, 20]. However, the Unyvero P55 system is only indicative of bacterial load in a semi-quantitative manner (+, ++ or +++ read-outs per target) and therefore this aspect was not further analysed here. Ozongwu et al. [19] noted an 11% partial/complete failure rate with the Unyvero P55 assay, a phenomenon also reported in earlier studies using the previous version of the assay [21,22,23,24]. In our study, however, only 1/74 (1.4%) of the Unyvero P55 tests was invalid, perhaps due to the difference between extraction of nucleic acids from BAL fluids compared to other specimen types such as sputa.

It was clear that routine BAL microbiological test results discussed on daily Microbiology ward rounds in the ICU contributed to patient management, with resulting changes to antimicrobials documented in 60% of cases. There were considerable potential benefits of using either molecular panel, in that true-negative and true-positive results would have been available much sooner in at least half of these cases, assuming a molecular assay turnaround time of 5 to 6 h compared to at least 24 h for preliminary culture results. However, negative results could not be relied upon due to the low sensitivity of the Unyvero P55 test and the small target range of the in-house bacterial PCR. In reality, therefore, if a molecular test was negative, results of culture would be awaited before any changes made. If a molecular test was positive, knowledge of the identity of the organism may have been sufficient to enable faster antibiotic decisions, as found in a previous BAL study in an ICU [25]. Given the low sensitivity and variable negative predictive value of Unyvero P55 antibiotic resistance gene detection, this aspect of the assay conferred no additional benefit and potentially some risk.

The strengths of this study were its use of a large and microbiologically well-characterised BAL specimen set, comparative molecular testing and corresponding detailed antimicrobial prescribing information. A previous study showed that Unyvero P55 enabled rapid antibiotic modification in nosocomial pneumonia, but was limited by low frequency of culture positivity, low numbers of specimens and BAL specimens in particular and lack of a molecular comparator test [26]. In that study, sensitivity and specificity were not specifically calculated, but Unyvero P55 identified significantly more bacterial co-infections than routine culture. A recent study of Unyvero P55 on BAL specimens in lung transplant recipients again had low sample numbers but reported a sensitivity of 66% with specificity of 100% as there were no additional PCR positive culture negative cases [7]. Given that this study was conducted in an outpatient setting, it was unclear how the results of molecular testing would have contributed to patient management and antibiotic resistance gene detection was not assessed [7]. A large UK study reported Unyvero P55 sensitivity of 89% and specificity of 95% on mainly sputa and endotracheal aspirates from a mixed population of CAP, HAP and VAP patients, but did not assess potential clinical impact [19].

The present study is limited however by its retrospective nature. Degradation and dilution of nucleic acids by storage may have reduced the pathogen load in the specimen set, giving a more severe test of sensitivity than in a prospective study with fresh specimens. However, on a target-specific basis, the sensitivity of the Unyvero P55 panel appeared to be lower than that of the in-house panel, indicating that the wider target range and ease of use may come at the expense of some sensitivity relative to in-house multiplex real-time PCR. Antibiotic prescription data and decision-making were also analysed retrospectively, although this information was well documented in clinical notes made at the time. Inferring potential benefits and risks is difficult retrospectively but we believe it gives a pragmatic insight into how useful these tests would be in our setting. It is possible that in a setting with a lower culture-positive rate, higher antibiotic resistance or lacking regular Clinical Microbiologist input, there may be greater benefits from the Unyvero P55 panel [26].

For now, there is still a clear need to perform traditional microbiological culture of BAL fluids from patients in the ICU, at the very least, to confirm negative molecular tests and to detect antibiotic resistance reliably. We are, however, still missing potentially 16% of cases of bacterial infection with culture alone, and possibly more where molecular targets are still required for particular bacterial species, so there is further scope to improve patient management and antibiotic stewardship. Exactly which molecular assay will prove the most useful in this regard will depend very much on the individual setting; in our hands, a narrower but more sensitive in-house PCR panel was a more pragmatic and cost-effective solution. It is clear that at present, however, molecular tests may only be a useful adjunct to routine testing.