Article Text
PDF
Abstract
Background A promising modality for diagnosing pulmonary manifestations of COVID-19 in the emergency department (ED) is point-of-care ultrasound (POCUS) of the lungs. The currently used PCR as well as chest X-ray and CT scanning have important disadvantages. The aim of this study is to evaluate the diagnostic accuracy of POCUS in patients with suspected pulmonary manifestations of COVID-19 in the ED.
Methods This prospective diagnostic accuracy study was conducted at the ED of our non-academic level 1 trauma centre (Isala, Zwolle, the Netherlands). Patients were enrolled between 14 April and 22 April 2020. Patients (aged ≥16 years) with suspected COVID-19 presenting to the ED underwent POCUS. All patients received current standard of care, including PCR (naso-oropharyngeal swab). Outcome of POCUS was compared with PCR or CT scan outcome to determine diagnostic accuracy. Diagnostic accuracy measures were calculated using 2×2 contingency tables.
Results 100 patients were eligible to participate in this study, data of 93 patients were analysed. 27 (29%) patients were found positive for COVID-19 by PCR or CT. POCUS had a sensitivity of 89% (95% CI 70% to 97%), specificity of 59% (95% CI 46% to 71%), negative predictive value of 93% (95% CI 79% to 98%) and positive predictive value of 47% (95% CI 33% to 61%). In a subgroup of patients without previous cardiopulmonary disease (n=37), POCUS had a sensitivity of 100% (95% CI 70% to 100%), specificity of 76% (95% CI 54% to 90%), negative predictive value of 100% (95% CI 79% to 100%) and positive predictive value of 67% (95% CI 41% to 86%).
Conclusion POCUS of the lungs could serve as a valuable, radiation-free tool for excluding pulmonary manifestations of COVID-19 in patients in the ED at the point of assessment, especially in patients without previous cardiopulmonary disease.
Trial registration Dutch Trial Register, No: NTR8544.
- ultrasound
- infectious diseases
- SARS
- respiratory
- pneumonia/infections
This article is made freely available for use in accordance with BMJ’s website terms and conditions for the duration of the covid-19 pandemic or until otherwise determined by BMJ. You may use, download and print the article for any lawful, non-commercial purpose (including text and data mining) provided that all copyright notices and trade marks are retained.
https://bmj.com/coronavirus/usageStatistics from Altmetric.com
Key messages
What is already known on this subject
The first studies on the use of point-of-care ultrasound (POCUS) for diagnosis of pulmonary manifestations of COVID-19 are promising, but until now studies addressing the diagnostic accuracy of POCUS for COVID-19 are lacking.
What this study adds
Overall sensitivity and negative predictive value of POCUS for pulmonary involvement of COVID-19 are relatively good at 89% and 93%, respectively.
In patients without a medical history of cardiopulmonary disease, POCUS may exclude pulmonary manifestations of COVID-19 (100% sensitivity, 100% negative predictive value, negative likelihood ratio 0).
Overall specificity of 59% and positive predictive value of 47% reflect that the pulmonary changes seen in COVID-19 overlap considerably with other lung pathologies.
Introduction
During the current COVID-19 pandemic, clinicians are searching for proper diagnostic instruments to diagnose or exclude SARS-CoV-2 infection in patients. Diagnostic evaluation with PCR has several disadvantages, such as delay to result availability (up to 8 hours) and the considerable risk of false negative outcomes.1 2 Other diagnostic tools, such as imaging modalities including chest X-rays and CT scans, are currently used as a single tool or combined with PCR for diagnosis of pulmonary manifestations of COVID-19.2 However, next to their own limitations regarding diagnostic accuracy, these have disadvantages such as capacity, logistic challenges and the use of ionising radiation.
Point-of-care ultrasound (POCUS) of the lungs may be used as a safer, non-invasive, radiation-free diagnostic tool for the pulmonary manifestations of COVID-19.3 4 POCUS is known for its high sensitivity and specificity in diagnosing pulmonary abnormalities such as pulmonary oedema (sensitivity 97%, specificity 95%) and pleural effusion (sensitivity and specificity >93%).5 6 Lung ultrasound is more accurate in quantifying pleural effusion than chest X-ray, and has similar accuracy for measuring pleural effusion as chest CT.6 7 Furthermore, POCUS can be performed as a bedside tool by the examining physician, which may provide important logistic and material benefits (eg, less use of personal protective equipment and cleaning) compared with X-ray and CT scanning.3
While the first studies on the use of POCUS for diagnosis of pulmonary manifestations of COVID-19 are promising, study sizes are small (up to 30 patients), and POCUS was only used in patients with proven COVID-19.4 8–11 Thus, the diagnostic accuracy of POCUS for COVID-19 is unknown.
The aim of this study was to determine the diagnostic accuracy of POCUS for the diagnosis of pulmonary manifestations of COVID-19 in patients presenting to the emergency department (ED).
Methods
Study setting and study design
This single-centre, prospective, diagnostic accuracy study was conducted at the ED of our non-academic level 1 trauma centre (Isala, Zwolle, the Netherlands). A convenience sample of 100 patients with suspected COVID-19 was enrolled between 14 April and 22 April 2020, during the first outbreak of the COVID-19 pandemic in the Netherlands.
This study was conducted in accordance with the amended Declaration of Helsinki. The institutional review board of Isala approved the protocol (protocol number NWMO 200414). Written informed consent was obtained from all patients. Our study was conducted and reported according to the Standards for Reporting Diagnostic Accuracy (STARD).12
Patient and public involvement
This research was done without patient involvement. Patients were not invited to comment on the study design and were not consulted to develop patient-relevant outcomes or interpret the results. Patients were not invited to contribute to the writing or editing of this document for readability or accuracy.
Study population
Both patients with COVID-19 as primary reason for ED referral and patients with another main complaint (eg, chest pain, trauma) who were suspected of COVID-19 were eligible for inclusion. Suspicion of COVID-19 was based on any of the following symptoms: fever ≥38°C, cough, dyspnoea, rhinorrhoea, anosmia, sore throat, diarrhoea or abdominal pain. Exclusion criteria were age <16 years, pre-existent heart failure, inability to perform ultrasound (eg, due to severe disease requiring other immediate interventions), or no consent.
Clinical trials evaluating the utility of POCUS findings in COVID-19 are scarce. The few trials that were conducted included a limited number of patients.4 8–11 Therefore in this trial a convenience sample of 100 was pursued.
Study protocol
POCUS was performed in the ED by one physician (SLH), who was blinded for all clinical information such as history, other diagnostic tests, outcomes and disposition. This physician (SLH) is an emergency medicine resident with 2.5 years of POCUS experience.
POCUS images were stored anonymously and reviewed by an emergency physician and expert sonographer (BYMK), who was blinded for all patient information and (diagnostic) outcomes. When differences in initial evaluation of POCUS were observed, final POCUS outcome (COVID-19 positive/negative) was based on consensus of both researchers.
POCUS was performed according to recent published COVID-19 POCUS protocols for use in the ED. Six zones per lung (a total of 12 zones) were scanned (per lung: two anterior, two axillary and two posterior zones).13 14 In patients who were unable to sit upright or lie on their side, the posterior zones were obtained as posterior as possible. A convex probe with a bandwidth of 2–6 MHz was used on a Philips Sparq POCUS system (Best, the Netherlands). The exam was performed using lung setting, with SonoCT, XRES and Autoscan turned off and depth set to 15 cm. POCUS was deemed COVID-19 positive when images in at least one zone met the criteria described in table 1. This clarification of POCUS findings in COVID-19 was based on the current available literature on POCUS for COVID-19 in the ED.13–15 Studies show that an irregular pleural line, multiple or confluent (waterfall) B-lines, subpleural consolidations and small pleural effusions are found in COVID-19. Pulmonary hepatisation and large effusions are not associated with COVID-19 but with other diagnoses according to the current literature.3 4 9 11
POCUS of the lungs scoring system for pulmonary manifestations of COVID-19
Final diagnosis of COVID-19 was defined as a positive PCR or a positive chest CT result for COVID-19 within 14 days of ED presentation. The CT scan was defined positive according to the COVID-19 Reporting and Data System (CO-RADS) classification when rated ‘4’ or ‘5’ (high or very high level of suspicion of COVID-19) and was obtained from the radiology report.16
All patients received standard workup in the ED which included arterial blood gas, laboratory testing, PCR (naso-oropharyngeal swab), chest X-ray or CT scanning. Treating physicians were not informed of ultrasound findings. Patients who were admitted to the hospital with a high clinical suspicion of COVID-19 were retested multiple times (naso-oropharyngeal swab) when the PCR result came back negative.
Outcome measures
The primary outcome was the diagnostic accuracy (ie, sensitivity, specificity, predictive values and likelihood ratios) of POCUS in patients suspected of COVID-19. For this, the POCUS outcome (COVID-19 positive/negative) served as an index test and diagnosis of COVID-19 by PCR or CT as a reference test. Secondary outcome was diagnostic accuracy of POCUS for pulmonary manifestations of COVID-19 patients without previous cardiac or pulmonary disease; in whom normally no pulmonary abnormalities are expected. Cases in which POCUS diagnosis was negative for pulmonary manifestations of COVID-19, but the PCR or CT result was positive, were described.
Data analysis
Distribution of continuous variables was tested using the Kolmogorov-Smirnov test with Lilliefors' correction. Continuous variables were represented as an average with SD or as a median with an IQR, depending on distribution. Nominal variables were displayed with absolute numbers and percentages. Missing data were reported. Diagnostic accuracy measures (sensitivity, specificity, predictive values and likelihood ratios) with corresponding 95% CI were calculated using 2×2 contingency tables. Inter-rater agreement for POCUS outcome was represented as percentage of agreement and measured by the Cohen’s kappa (κ) test. A κ value <0 indicated no agreement, 0–0.20 slight, 0.21–0.40 fair, 0.41–0.60 moderate, 0.61–0.80 substantial and 0.81–1 almost perfect agreement.17 All statistical analyses were done using IBM SPSS Statistics Premium V.24 for Windows.
Results
In total, 100 patients with suspected COVID-19 were included. Five patients with proven COVID-19 were excluded from analysis, because diagnostic information was exposed to the physician performing ultrasound (‘proven COVID-19’ was mentioned in the reason for referral). In one patient ultrasound images were not assessable due to poor image quality, in another patient a reference test was lacking (no PCR or CT obtained). Both patients were excluded. In total, 93 patients were analysed (figure 1, table 2).
Baseline criteria of patients diagnosed both COVID-19 positive and negative by PCR or CT scanning
Patient flow chart. COVID-19 positive was defined as PCR or CT positive for COVID-19 (two patients had the same test result on both PCR and CT). *Excluded because of known diagnosis of COVID-19 prior to inclusion. POCUS, point-of-care ultrasound; n, number.
A PCR was performed in all patients, CT scanning in 26 (28%) patients. In total, 29% (n=27) of the patients were COVID-19 positive based on PCR (n=24) or CT scanning (n=5; table 3). POCUS was performed in all 93 patients (figure 2, online supplemental file 1) and was considered positive for pulmonary manifestations of COVID-19 in 55% of the patients (n=51). Inter-rater agreement on POCUS outcome was found in 87% (n=81), κ=0.74. POCUS had a sensitivity of 89% (95% CI 70% to 97%), specificity of 59% (95% CI 46% to 71%), negative predictive value (NPV) of 93% (95% CI 79% to 98%), positive predictive value (PPV) of 47% (95% CI 33% to 61%), negative likelihood ratio of 0.19 (95% CI 0.06 to 0.56) and a positive likelihood ratio of 2.17 (95% CI 1.58 to 2.99, table 4 and online supplemental table 1).
Supplementary video
Supplemental material
Ultrasound abnormalities found with POCUS in patients in this study. (A) Irregular pleural line with small subpleural consolidation and confluent B-lines. (B) Irregular pleural line with confluent B-lines (waterfall B-line). (C) Subpleural consolidation with multiple B-lines. POCUS, point-of-care ultrasound.
Outcome of POCUS for COVID-19 in comparison to PCR and CT
Diagnostic accuracy of POCUS for pulmonary manifestations of COVID-19
Presence or absence of respiratory symptoms did not significantly alter the aforementioned outcomes (online supplemental table 2). POCUS in patients without previous cardiac or pulmonary medical history (n=37) had a sensitivity of 100% (95% CI 70% to 100%), specificity of 76% (95% CI 54% to 90%), NPV of 100% (95% CI 79% to 100%), PPV of 67% (95% CI 41% to 86%), negative likelihood ratio of 0 and a positive likelihood ratio of 4.17 (95% CI 2.07 to 8.37, table 4 and online supplemental table 3).
Supplemental material
Supplemental material
In total, POCUS missed three patients with COVID-19. One patient was diagnosed by PCR and had pulmonary complaints, normal vital signs and no arterial blood gas or chest X-ray abnormalities. This patient was discharged home after ED visit. The other two patients were labelled COVID-19 positive based on the result of the CT scan, while their consecutive PCR tests were negative. Both patients were rated ‘4’ on the CO-RADS classification (high level of suspicion of COVID-19).16 In the first patient, POCUS of the posterior zones was not optimally performed due to a traumatic injury and most abnormalities on CT were found in the posterior lung fields. This patient was determined COVID-19 negative during hospital admission due to uncertainty about the CT result, two negative PCR tests, and in retrospect no clear clinical symptoms of COVID-19. In the other patient, who presented with pulmonary complaints, the radiology report described non-typical COVID-19 abnormalities that may be due to a history of fibrotic lung disease. In retrospect, POCUS did show an irregular pleural line in 1 of the 12 views. PCR was performed three times and remained negative.
Discussion
POCUS had a sensitivity of 89%, specificity of 59% and a NPV of 93% compared with PCR or CT scanning in diagnosing pulmonary manifestations of COVID-19. Moreover, in patients without cardiopulmonary disease, POCUS had a 100% sensitivity and a 100% NPV for pulmonary involvement of COVID-19. This indicates that pulmonary manifestations of COVID-19 may be excluded by POCUS in patients without previous cardiac or pulmonary medical history; in whom normally no pulmonary abnormalities are expected. This was underlined by a negative likelihood ratio of 0.19 and 0 in a general population and in a population without cardiopulmonary history, respectively. These numbers contribute to a useful change in post-test probability, as opposed to the less contributing positive likelihood ratio (2.17 and 4.17, respectively).
The difference in sensitivity and specificity between patients with and without cardiopulmonary disease may very well be related to the non-specific lung findings of COVID-19 observed with POCUS.6 Whereas multiple B-lines are seen in COVID-19, they can also be found in pulmonary oedema due to cardiovascular disease, aspiration, acute respiratory distress syndrome, interstitial lung disease or pneumonia.18 Furthermore, subpleural consolidations and effusions are observed in both COVID-19 as well as in other viral and non-viral pneumonia and pulmonary embolism.6 18 19 Also, irregularity of the pleura may be observed in other pulmonary diseases as well.6 Therefore, abnormal POCUS findings in patients with previous cardiopulmonary disease may be related to these previous diseases as well as to COVID-19, or even another disease or condition. Distinguishing the cause may therefore be difficult. For this reason patients with known heart failure were not enrolled in this study. Inclusion of these patients would have probably led to a lower diagnostic accuracy. However, in patients with no cardiopulmonary medical history, abnormalities are not expected in advance. Hence, the diagnostic accuracy should be improved, as observed in this study.
In total, three patients diagnosed with COVID-19 were missed by POCUS. These cases illustrate that POCUS might be normal in patients with mild disease, or when pulmonary manifestations of COVID-19 are absent. Furthermore, it underlines the importance of visualising the posterior lung fields. Another explanation for the negative POCUS outcome in the two patients diagnosed by CT only, who had two and three negative PCR tests, is a false positive CT scan for COVID-19.
This was a single-centre study, conducted in an ED; the results may not be generalisable to other settings such as patients in an intensive care unit. Furthermore, the high prevalence of COVID-19 in this study (29%) could lead to a spectrum effect. In a population with a lower incidence of COVID-19 POCUS may lead to more false negatives.
As this was a single operator study, the test performance may be different in other hands. However, the sonographer had a moderate amount of experience (2.5 years); thus, characteristics may not be strictly limited to highly skilled providers.
In this study, we considered a positive PCR or CT scan to be the reference test for diagnosing COVID-19. While this was current practice in the Netherlands during the conduction of this study, it has important limitations such as low sensitivity and therefore missed diagnoses of COVID-19. The reported sensitivity and specificity of PCR and CT differ tremendously.20 Sun et al and Adams et al both conducted a systematic review in which the CT sensitivity differs from 80% to 98%, although it lacks specificity (25% to 72%).1 20 21 This low specificity results in a high false positive rate that might be caused by other pneumonia-causing factors. PCR has a sensitivity ranging from 37% to 91%.22 23 Specificity of most of the PCR tests is 100%; hardly any false-positive results occur.24 Chest X-ray has a sensitivity of 69% detecting lung abnormalities in suspected patients with COVID-19, which is significantly lower compared with PCR or CT.22 25
Only three patients in this study were diagnosed by CT only. Still, it is possible that the reported diagnostic accuracy of POCUS is currently underestimated, which was also seen in the increase of diagnostic accuracy when compared with PCR only as a reference test.
This study investigated pulmonary manifestations of COVID-19. It is important to realise that a negative POCUS result for pulmonary involvement of COVID-19 does not exclude the presence of a SARS-CoV-2 infection. Furthermore, because this study represents a convenience sample of limited sample size, no definitive conclusions can be made.
This study shows that POCUS could probably exclude significant pulmonary involvement of COVID-19 in patients without previous cardiopulmonary disease. POCUS could possibly be used as a triage tool for pulmonary manifestations of COVID-19 during the pandemic when resources are overwhelmed, but more research is needed.
In conclusion, POCUS of the lungs could serve as a valuable, easy accessible and radiation-free tool for excluding pulmonary manifestations of COVID-19 in patients in the ED at the point of assessment, especially in patients without previous cardiopulmonary disease.
Abstract translation
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.Acknowledgments
We would like to thank all the patients who took part in the study. Furthermore, we thank the emergency department healthcare workers for their cooperation. Finally, we would also like to thank Emile van der Have (Emergency Physician, Medical Centre Leeuwarden, the Netherlands) and Vincent Rietveld (Emergency Physician, Dijklander Hospital Hoorn, the Netherlands and founder of Dutch Emergency Ultrasound (DEUS)) for their intellectual input regarding the POCUS scanning protocol.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Handling editor Katie Walker
Twitter @Britt_NL
Contributors SLH, BYMK and IJER contributed to the data collection. HL contributed to the data analysis. SLH, HL, LCJ and BYMK contributed to the development of the study design and interpretation of the data. LCJ, IJER and HL contributed to the literature search. All authors critically revised the manuscript. All authors contributed substantially to writing, editing, revising and finalising the manuscript before submission. All authors approved the final version of the manuscript.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement Data are available upon reasonable request. The research protocol (written in Dutch) and database are available upon request. Requests can be sent to SL Haak (s.l.haak@isala.nl). Reuse is permitted after consultation.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.