Abstract
Respiratory syncytial virus is a common cause of bronchiolitis. Historically, point-of-care tests have involved antigen detection technology with limited sensitivity. The aim of this study was to prospectively evaluate the diagnostic accuracy and model the economic impact of the Roche cobas® Liat® point-of-care influenza A/B and respiratory syncytial virus test.
The “DEC-RSV” study was a multi-centre, prospective, observational study in children under 2 years presenting with viral respiratory symptoms. A nasopharyngeal aspirate sample was tested using the point-of-care test and standard laboratory-based procedures. The primary outcome was accuracy of respiratory syncytial virus detection. The cost implications of adopting a point-of-care test were modelled using study data.
A total of 186 participants were recruited, with both tests performed on 177 samples. The point-of-care test was invalid for 16 samples (diagnostic yield 91%) leaving 161 available for primary analysis. After resolving discrepancies, the cobas® Liat® respiratory syncytial virus test had 100.00% (95% CI 96.07%–100.00%) sensitivity and 98.53% (95% CI 92.08%–99.96%) specificity. Median time to result was 0.6 h (interquartile range (IQR) 0.5–1) for point-of-care testing and 28.9 h (IQR 26.3–48.1) for standard laboratory testing. Estimated non-diagnostic cost savings for 1000 patients, based on isolation decision-making on point-of-care test result, were £57 010, which would increase to £94 847 when cohort nursing is used.
In young children the cobas® Liat® point-of-care respiratory syncytial virus test has high diagnostic accuracy using nasopharyngeal aspirates (currently an off-licence sample type). Time to result is clinically important and was favourable compared to laboratory-based testing. The potential exists for cost savings when adopting the point-of-care test.
Abstract
This prospective evaluation of the cobas Liat point-of-care RSV test in children demonstrated high diagnostic accuracy using nasopharyngeal aspirate samples, with favourable time to result compared to usual laboratory-based testing procedures https://bit.ly/2yKKmUB
Introduction
Respiratory syncytial virus (RSV) is a ubiquitous and highly transmissible respiratory tract pathogen that is the most common cause of bronchiolitis in infants and young children [1]. Bronchiolitis is often diagnosed clinically [1]. Testing for RSV may, however, facilitate the rationalisation of antibiotic use, reduce additional diagnostic investigations and inform caregivers and clinicians of a cause for a child's symptoms [2]. A positive result can facilitate targeted isolation measures to prevent cross-infection or enable cohort nursing of a group of infected children [3]. Several anti-RSV therapies are currently under evaluation in clinical trials, and there is likely to be a future requirement for prompt and accurate detection of RSV to allow targeted treatments to be delivered effectively [4]. However, current clinical guidelines for the management of bronchiolitis only recommend testing in high-risk patients [5, 6], to facilitate cohorting [7, 8], or do not mention testing [9], partly due to a lack of high-quality evidence in this area [1, 9].
If laboratory-based testing for respiratory viruses is performed, it may take over 24 h before results are available. Historically, point-of-care (POC) systems for RSV were based on antigen detection tests. The sensitivity of antigen detection tests for RSV is dependent on viral load and they are therefore of limited clinical utility [10–12]. More recently, polymerase chain reaction (PCR)-based POC testing systems for RSV and other respiratory viruses have been developed, offering improved sensitivity and specificity [11, 13–16]. One example is the Roche cobas® Liat® system (hereafter referred to as the cobas® Liat® POC test), which can produce a result for influenza A/B and RSV within 20 minutes [17].
The cost-effectiveness of POC tests is a crucial consideration in their ultimate adoption in healthcare systems [18, 19]. Previous studies have assessed the implications of introducing POC tests for influenza and RSV at the hospital entry-point for adults [20–22]. In paediatrics a before/after study design has been used to estimate the impact of introducing a POC test on oseltamivir prescribing and health-care costs [23]. Prospective clinical and economic evaluations have yet to examine whether the increase in diagnostic accuracy and faster time to result provided by POC tests could impact on paediatric patient management and possible cost savings.
The primary aim of the “DEC-RSV” study was to prospectively assess the diagnostic accuracy of the RSV component of the cobas® Liat® POC test as compared with standard laboratory-based tests, using nasopharyngeal aspirate samples from children presenting with viral respiratory symptoms. Time to clinically actionable result, likely impact on hospital isolation resources and potential cost implications were also assessed.
Methods
Ethical approval
The study was approved by the East Midlands–Leicester Central Research Ethics Committee (16/EM/0456). Written informed consent was obtained from a caregiver and followed Good Clinical Practice [24]. The study sponsor was The Newcastle upon Tyne Hospitals NHS Foundation Trust (NuTH), and it was National Institute for Health Research portfolio adopted (CPMS ID 32674).
Study design
Participants
A prospective study was conducted between November 2017 and May 2018 at the Great North Children's Hospital (GNCH), NuTH, and Sunderland Royal Hospital (SRH), South Tyneside and Sunderland NHS Foundation Trust, UK. GNCH is a regional centre providing secondary general and tertiary paediatric speciality services. SRH is a secondary care general paediatric unit. Children under 2 years were eligible if they presented with an acute respiratory illness that clinicians considered could be due to RSV infection and if sampling was not contraindicated (see figure 1 for study process). The design was informed by a separate pilot study conducted in winter 2016–2017.
Sampling and testing
A nasopharyngeal aspirate sample was obtained and tested using the cobas® Liat® POC test in a ward setting and via the usual laboratory-based system (see supplements 1 and 2 for details and standard operating procedures). The study was observational, so neither the caregivers nor the clinicians were informed of the POC results. The results were anonymised and uploaded to a REDCap Cloud-based database (Version .5, nPhase, Encinitas, CA, USA).
Outcomes and sample size estimation
The primary outcome was the sensitivity, specificity, and positive and negative predictive values of the RSV component of the cobas® Liat® POC test. Secondary outcomes included time to actionable result, influenza A and B concordance, clinical impact and the cost implications of introducing rapid RSV testing in NHS paediatric acute care facilities. With a 25% prevalence of RSV (estimated in an audit of young children at GNCH in winter 2015–2016), 195 participants would provide 90% power to distinguish a specificity of 98% versus a 93% lowest threshold (lowest acceptable specificity for the cobas® Liat® POC test based on analytical validation studies) [25].
Discrepancy analysis
When the laboratory test results became available, the study nurses compared them with the cobas® Liat® POC result. If there was a discrepancy, the remaining sample was tested at the other site using the alternate laboratory-based test (see supplement 3).
Statistical analysis
Concordance tables were constructed (before and after resolving discrepancies) in Microsoft Excel. All comparative analyses were carried out in JMP® Pro V13.0.0 (JMP, Marlow, UK) and RStudio V0.99.446 (RStudio Inc, Boston, MA, USA).
Clinical impact
Participants’ admission and discharge diagnoses, length of stay, infection control procedures (i.e. barrier nursing or single room isolation), antimicrobial treatment and other diagnostic procedures performed were recorded. The time to actionable result was defined as that between sample collection and the results of the laboratory test becoming available to clinicians or, for the cobas® Liat® POC test, the time between sampling and result being displayed. At both sites, the standard clinical procedure was to perform laboratory testing in batches.
Cost analysis
A model was developed in TreeAge-pro-2017 (TreeAge Software, Williamstown, MA, USA) to extrapolate the results of the study to a hypothetical cohort of 1000 patients presenting to a UK National Health Service (NHS) hospital during the 2017–2018 winter season (see supplement 4 for details). The parameters for modelling were quantified with frequency and usage data from the study, drug costs from Monthly Index of Medical Specialities and the British National Formulary [26, 27], and isolation costs from the NHS Reference costs, HRG codes PD15A-D (see supplementary table S5) [28].
Results
Recruitment and RSV prevalence
One hundred and eighty-six patients were recruited, 105 at GNCH and 81 at SRH. One participant was discharged prior to sample collection or clinical data recording. Data from the remaining 185 participants were analysed. The target sample size of 195, based on an RSV prevalence of 25%, was not reached. However, with an RSV prevalence of 57%, the study remained adequately powered.
Participant demographics
The mean age of participants was 5.7 months (table 1). Bronchiolitis was the most common admission and discharge diagnosis. Median length of stay differed between sites: 90.8 h (interquartile range (IQR) 51.0–184.3) at GNCH and 45.6 h (IQR 22.1–68.1) at SRH. There was greater use of high-flow nasal cannula oxygen, antibiotics and nasogastric feeding at GNCH.
Diagnostic testing results
Figure 2 summarises the results of testing in the study. The RSV results for the reference laboratory test were unavailable for six participants and the cobas® Liat®POC results not recorded for another two. The data from these eight participants were not included in the diagnostic accuracy analysis. For 34 participant samples, the cobas® Liat® POC test produced an invalid result (32) or an error message (two) on the first attempt. These samples were retested following the protocol (see figure 1 and supplement 2), 18 of which produced a result and 16 were unsuccessful on a second attempt, giving an overall 7.6% (16/211) test failure rate and a diagnostic test yield of 91.0% (161/177). Data from 161 participants were available for comparative diagnostic accuracy analysis.
Disagreement between the cobas® Liat® RSV result with the laboratory tests occurred for five samples. Discrepancy resolution testing was carried out for four of these samples (see supplementary material supplement 5), allowing data from 160 participants to be used for diagnostic accuracy analysis following discrepancy resolution.
Diagnostic accuracy
For the primary analysis, we compared the cobas® Liat® POC test results (first or single repeat) with the results from the reference laboratory tests (summarised in table 2). The combined site sensitivity for RSV detection (including repeat) was 98.90% (95% CI 94.03%–99.97%) and the combined site specificity was 94.29% (86.01%–98.42%). The overall study positive and negative predictive values were 95.74% (89.68%–98.30%) and 98.51% (90.37%–99.78%) respectively.
Reclassification after discrepancy analysis
The test results after discrepancy analysis are detailed in table 3. Following discrepancy testing, two samples initially classified as false positives were reclassified as true positives, one false positive was confirmed as a false positive, and one false negative was reclassified as a true negative. The remaining sample, classified as false positive, was not retested and therefore removed from the re-analysis (see supplement 5 for further details).
The sensitivity and specificity for RSV detection (both sites combined, including repeat testing and following discrepancy resolution) were 100.00% (96.07%–100.00%) and 98.53% (92.08%–99.96%) respectively. The positive and negative predictive values were 98.92% (92.93%–99.84%) and 100.00% (94.58%–100.00%) respectively.
Influenza A and B concordance analysis
Six samples tested positive with the reference tests for influenza A and were correctly categorised by the cobas® Liat® test. There were no positive influenza B samples. One RSV positive sample was incorrectly classified as positive for influenza A (false positive), influenza B (false positive) and RSV (true positive) on the cobas® Liat® system (see tables 4 and 5). The specificities for influenza A and B detection were 99.35% (95% CI 96.44%–99.98%) and 99.38% (95% CI 96.57%–99.98%) respectively.
Clinical impact
Time to result
The median time for the laboratory-based test results to be available to clinicians was 28.9 h (IQR 26.3–48.1) across both sites. For GNCH (with onsite laboratory testing) it was 27.9 h (IQR 25.6–48.0) and for SRH (with offsite laboratory testing) 29.7 h (IQR 27.9–49.0). The median time for the cobas® Liat® POC results to be available for clinicians (from when the sample was taken) was 0.6 h (IQR 0.5–1.0). For GNCH it was 0.7 h (IQR 0.5–1.0) and for SRH it was 0.6 h (IQR 0.5–0.90).
Isolation practice
Data on isolation practice were available for 156 participants. Almost all participants (148/156) were placed in single room isolation, as per site protocol for patients presenting with acute respiratory illnesses, and 97% of those with confirmed RSV were isolated. Personal protective equipment (PPE) is used as standard in SRH; therefore, this was not recorded in addition to single room isolation. In GNCH, outside of a pandemic, PPE is only used when there is confirmation of an infective cause for a child's symptoms or in high-risk cases and was used for 92% of participants.
Cost analysis
If a POC RSV test were to inform the use of isolation facilities (i.e. a negative result would allow management in an open ward setting) for 1000 patients, 100 patients would have received an invalid result and therefore would have been managed according to standard care (57 assumed RSV positive and isolated, 43 assumed RSV negative and not isolated); 519 of the remaining 900 patients would have tested positive and been immediately isolated with PPE use (986.1 isolation bed days). Six of these patients would have been incorrectly classified as positive (false positives) and thus would have been unnecessarily isolated for a total of 11.4 days. The remaining 380 patients who would have tested negative would be managed in an open ward. According to the frequency data recorded in the study, PPE would have been invoked for 27% of these (103 patients or 195 total days of PPE use).
The time and cost analysis results are presented in table 6. On average, each patient would spend 0.47 fewer days in single room isolation with use of the POC RSV test than with standard laboratory testing. The total costs of bed days and barrier nursing are lower with the POC test; treatment costs are the same for both testing strategies. For a cohort of 1000 paediatric patients, use of the cobas® Liat® POC test to direct use of isolation facilities would cost £57 010 less than standard testing (the cost of the diagnostic tests including staff time and laboratory overheads have been excluded from this analysis to allow generalisability of the results to other manufacturers and laboratory setup).
Sensitivity analysis of variables in cost analysis
Average length of stay and the cost of a day in single bed isolation were the most influential variables in the cost analysis. Threshold analysis indicates that if single bed isolation rooms were reduced in cost by 20%, then use of the cobas® Liat® POC test for RSV would be more expensive than a standard laboratory testing strategy.
If the cost of the single-use assays were included in this analysis (without accounting for the cost of implementation of instruments), the cobas® Liat® Influenza A/B and RSV assay would need to cost <£124 for it to provide cost savings to the NHS provider (based on a standard laboratory test costing £74).
Cohort nursing of RSV positive patients
In situations where there is pressure on isolation rooms, and the incidence of RSV is high, e.g. in peak winter months, clinicians may choose to cohort all known RSV positive patients in bay areas. If all RSV positive patients were cohorted in a six-bed bay, a further £94 847 savings could be realised for a cohort of 1000 patients (see table 6 for breakdown of costs).
Discussion
In this prospective study, we found the RSV component of the cobas® Liat® POC test to have good diagnostic accuracy with high positive and negative predictive values using laboratory-based PCR testing as the gold standard. These high diagnostic accuracy results in which nasopharyngeal aspirates were used are similar to those from published studies of RSV POC tests in which nasal swabs (fresh or frozen) were used [14, 16]. The cobas® Liat® POC test is not currently indicated for use with nasopharyngeal aspirate samples and was developed for use with nasopharyngeal swabs. Nasopharyngeal aspirates are commonly used in the UK for respiratory virus testing in young children. Although not statistically powered for assessing the accuracy of testing for influenza A, in the few positive cases included in the study, the cobas® Liat® POC test showed good agreement with the reference test, and our results suggest high specificity for influenza A and B. A limitation of our study design was the assumption that the reference laboratory-based test was 100% sensitive and specific. Because this is unlikely to be the case, we investigated all discrepancies between the reference test and the cobas® Liat® POC test.
Time to result is a major consideration in the clinical utility of a test and, as illustrated in our study, is a drawback of standard laboratory-based testing strategies that involve transporting samples to an offsite facility and/or batch testing. A short time to actionable result maximises the impact of a test on clinical management, including decision-making around antibiotic prescribing, performing additional investigations, cohort nursing and isolation precautions. We found the median time to result, including sample collection time, to be substantially shorter, at 36 min for the cobas® Liat® POC test, compared to the standard laboratory-based testing procedure of around 29 h at our study sites. It follows that an accurate and rapid POC test for RSV could be especially useful in an Emergency Department or Assessment Unit [3, 12]. In this study, POC testing was performed by clinical research (non-laboratory) staff in the ward. We experienced an overall test failure rate of 7.6%. The study protocol involved diluting the sample with viral transport medium on repeat testing should there be an initial test failure.
A number of novel anti-RSV therapies are currently under evaluation in clinical trials [4]. If they become available for clinical use, the ability to diagnose RSV rapidly and accurately will be crucial because such therapies are likely to be active against only RSV specifically, most effective in the early phases of an illness and expensive. Targeted treatment will be required to maximise efficacy while minimising costs and adverse effects in children who would not benefit from the treatment.
Data collected in the study allowed detailed modelling and analysis of the costs associated with diagnosis and treatment of suspected RSV guided by POC testing. We estimated non-diagnostic cost savings with the cobas® Liat® POC test for RSV to be >£50 000 per 1000 patients, and this could be increased to around £100 000 if the POC test were to guide cohort nursing. Length of stay in hospital, the cost of a day in single room isolation and RSV prevalence had the greatest influence on the results of the cost analysis, indicating that the potential for cost savings would vary with seasonality and hospital resources. A limitation of our analysis was the assumption that the POC testing result would direct fully use of isolation resources. Decisions about isolation are often multifactorial and rely on clinical judgement in combination with the results of investigations [29]. If only specific viruses are tested for by a POC system, other highly transmissible viruses cannot be ruled out. Positive results would then indicate cohort nursing, but a negative result would not mandate stepping down of isolation precautions.
The adoption of a POC test for RSV is also likely to have indirect effects on patient and information workflows, encouraging optimal use of healthcare resources and reduction of nosocomial infection. Such effects have been demonstrated in UK hospitals implementing molecular POC tests for influenza [30]. An interventional study with associated cost-effectiveness analysis would be required to fully capture the impact of such a test. The indirect health and cost implications of nosocomial transmission of RSV to other patients were also not included in our model. We observed a reduction in the prescription of new courses of antibiotics once the laboratory-based test results were available to clinicians (at a median of 28.9 h after sampling). If implemented clinically, POC test results would likely be available considerably earlier and be associated with additional cost savings and patient benefits from reductions in unnecessary antibiotic prescriptions.
In summary, we found the cobas® Liat® POC test to have high sensitivity and specificity for the detection of RSV in nasopharyngeal aspirate samples from children under 2 years presenting with symptoms of a respiratory viral infection. The high diagnostic accuracy and favourable time to result suggest that the cobas® Liat® POC test could provide a useful aid to clinical decision-making and potentially lead to cost savings, especially in hospitals with ward bays and a limited number of isolation rooms. These benefits must be balanced against the fact that many modern hospitals are designed with large numbers of individual patient rooms, making cohort nursing of infants positive for RSV unnecessary. In the future, when specific anti-RSV therapies become available, accurate rapid tests for RSV will become more important clinically.
Supplementary material
Supplementary Material
Please note: supplementary material is not edited by the Editorial Office, and is uploaded as it has been supplied by the author.
Supplementary material 00018-2020.supp
Acknowledgements
We thank the families who agreed for their children to take part, and all clinical, laboratory and research staff across both sites who supported the study.
Footnotes
This article has supplementary material available from openres.ersjournals.com.
Data collected for this study will be made available (in the form of any or all from the de-identified data on the study database, study protocol, statistical analysis plan and analytic code) to researchers who provide a methodologically sound research proposal, to assist with achievement of aims in the approved proposal. Data will be available from the time of publication of the article in print. Proposals should be directed to the corresponding author.
Support statement: Roche Diagnostics provided equipment and consumables, and funded the research costs. The study was performed and analysed independently by the National Institute for Health Research (NIHR) Newcastle In Vitro Diagnostics Co-operative. AJA, CL, JS, SG and AJS are funded by the NIHR Newcastle In Vitro Diagnostics Co-operative. The views expressed are those of the authors and not necessarily those of the NIHR or the Department of Health and Social Care. MB was supported by a Medical Research Council Clinician Scientist Fellowship (MR/M008797/1). Funding information for this article has been deposited with the Crossref Funder Registry.
Conflict of interest: A.J. Allen has nothing to declare.
Conflict of interest: A. Gonzalez-Ciscar has nothing to declare.
Conflict of interest: C. Lendrem has nothing to declare.
Conflict of interest: J. Suklan has nothing to declare.
Conflict of interest: K. Allen has nothing to declare.
Conflict of interest: A. Bell has nothing to declare.
Conflict of interest: F. Baxter has nothing to declare.
Conflict of interest: S. Crulley has nothing to declare.
Conflict of interest: L. Fairlie has nothing to declare.
Conflict of interest: D. Hardy has nothing to declare.
Conflict of interest: L. Johnston has nothing to declare.
Conflict of interest: J. McKenna has nothing to declare.
Conflict of interest: N. Richards has nothing to declare.
Conflict of interest: G. Shovlin has nothing to declare.
Conflict of interest: C. Simmister has nothing to declare.
Conflict of interest: S. Waugh has nothing to declare.
Conflict of interest: P. Woodsford has nothing to declare.
Conflict of interest: S. Graziadio has nothing to declare.
Conflict of interest: M. Power has nothing to declare.
Conflict of interest: A.J. Simpson has nothing to declare.
Conflict of interest: P. Kumar has nothing to declare.
Conflict of interest: K. Eastham has nothing to declare.
Conflict of interest: M. Brodlie reports grants and speaker fees paid to Newcastle University from Roche Diagnostics during the conduct of the study; and grants from Pfizer, speaker fees paid to Newcastle University from Novartis and TEVA, and travel expenses for educational meetings from Boehringer Ingelheim and Vertex Pharmaceuticals, outside the submitted work.
- Received January 13, 2020.
- Accepted May 22, 2020.
- Copyright ©ERS 2020
This article is open access and distributed under the terms of the Creative Commons Attribution Non-Commercial Licence 4.0.