Abstract
Background We describe implementation of a clinical decision support system, a computer-guided consultation (CGC), in the assessment of subjects referred with suspected obstructive sleep apnoea syndrome (OSAS).
Methods Two cohorts of patients were assessed. The first 100 cases had data collected with the CGC by a specialist sleep physician (stage1). A further 100 cases were assessed by a nonspecialist using the CGC (stage 2). For each case, the diagnosis suggested by the CGC was compared with the final diagnosis made by a second specialist sleep physician blinded to the CGC diagnosis.
Results Stage 1: of 100 people evaluated, a final diagnosis of OSAS was made by both the sleep specialist and CGC in 88% of cases. In 7 of the remaining 12 cases, both agreed there was “No evidence of OSAS”; in 5 cases the CGC did not reach a final diagnosis instead prompting specialist referral. Stage 2: 100 people were evaluated; 95% were evaluable. Both CGC and the sleep specialist made a diagnosis of OSAS in 83 cases (87%), in 5 cases both agreed there was no OSAS, whereas in 7 cases the CGC prompted a specialist review due to unexplained symptoms. The CGC was concordant with the final diagnosis in 95% and 93% of cases in the two cohorts, respectively and where there was doubt, prompted for clinical review. No OSAS cases were overlooked by the CGC.
Conclusion An intelligent CGC program creates opportunities in sleep medicine management pathways to safely yet effectively utilise nonspecialists working under specialist supervision.
Abstract
A clinical decision support system may be used safely to assess subjects with suspected obstructive sleep apnoea syndrome when used by specialists and nonspecialists alike https://bit.ly/2AFPD0r
Introduction
Obstructive sleep apnoea (OSA) is a common condition, estimated to occur in 14% of men and 5% of women, characterised by the cessation of breathing due to repetitive narrowing of the upper airway during sleep with the resulting fragmentation of sleep leading to excessive daytime sleepiness [1, 2]. Obstructive sleep apnoea syndrome (OSAS), characterised by the presence of OSA along with excessive daytime sleepiness (EDS), has been associated with a range of cardiovascular morbidities including hypertension, coronary artery disease and atrial fibrillation [3–5]. OSAS has been associated with an increased risk of road traffic accidents, is linked to greater absenteeism from work, excess lost workdays, disability claims, workplace-related accidents and decreased productivity [6–16]. In those patients diagnosed with OSAS, treatment with continuous positive airways pressure (CPAP) leads to improvement in daytime sleepiness, quality of life [17] and concentration translating into a reduction in motor vehicle accidents as well as benefiting work performance [18–20].
The increasing prevalence of OSA in society threatens to overwhelm healthcare systems already struggling to balance capacity, demand and cost. The use of technology in healthcare such as a clinical decision support system (CDSS) may help to address these challenges. As clinical practice moves to paperless formats, several opportunities emerge with one being the implementation of CDSSs as components or as tools available alongside an electronic case sheet. Furthermore, there is the possibility of creating a comprehensive guided consultation, as reported here, which supports the entire diagnostic and management process. We have developed and reported that such a guided consultation works for chronic obstructive pulmonary disease (COPD) care for both nurses and doctors [21]. The computer-guided consultation (CGC) is a comprehensive electronic consultation with multiple clinical decision support algorithms, which together prompt the clinician through the whole management of the condition, in this case OSA. It could be regarded as an evolved CDSS as it forms a full clinical record, supports the entire assessment process with multiple algorithms aiming to reach a correct diagnosis based on the history, prompted and interpreted investigations and then to suggest a management plan. We describe its use suspected OSA and assess whether it is effective and safe when used in diagnosing patients referred to a specialist sleep service with suspected OSAS.
Methods
At University Hospital Aintree, a regional tertiary sleep medicine centre, we implemented a CGC in the clinical pathway for the assessment of subjects referred by their general practitioner with suspected OSAS as part of a service redesign. The implementation was approved by our sleep service clinical lead, sleep service business manager and hospital information technology lead. The diagnostic accuracy of the CGC is comparable with what was current standard practice (i.e. a specialist physician assessment) in a phased two-stage implementation process in the sleep service, firstly when the CGC is used by a physician and secondly, when used by a nonclinician with minimal background training in the field of sleep medicine.
The computer-guided consultation
The CGC is hosted on a local UK National Health Service (NHS) server and is password protected enabling Caldicott principles and General Data Protection Regulations to be satisfied [22]. This ensures that patient data gathered by the CGC is duly and lawfully protected according to principles set in a national framework and that these data are only used when it is appropriate to do so, with anonymity being preserved. The CGC in addition, enables all patients to have a standardised electronic record of their condition, which can be used for future care and can be customised to local guidance priorities, while not excluding individual management. All patients gave individual consent to the use of and holding of their data.
The CGC software is structured and guides the operator and the patient through a number of sections:
History: the history is tailored to sleep medicine and incorporates embedded algorithms taking into account factors that may influence the presentation of subjects with suspected OSA such as sleep hygiene, occupation and the influence of comorbidity, particularly those conditions pertinent to sleep-disordered breathing. The software highlights those conditions that may increase the likelihood of underlying OSA in an individual, such as atrial fibrillation and refractory hypertension [3, 4]. The software not only integrates the key symptoms of OSAS, including more rare clinical presentations of OSAS but also incorporates into the algorithms those symptoms that may be indicative of other common sleep disorders, such as restless legs syndrome, shift-work syndrome and disorders of circadian rhythm. In addition, the detection of sleep hygiene-related issues is specifically included in the CGC algorithms and this includes management strategies of poor sleep hygiene and insomnia should this be detected. Similarly, the software also takes into account the presence of any nasal symptoms contributing to the subject's clinical presentation.
Clinical examination: includes description of key upper airway anatomy relevant to the OSA patient (such as craniofacial abnormalities and tonsillar enlargement), measurement of blood pressure (BP) and calculation of body mass index (BMI) and Epworth sleepiness score (ESS) [23]
Sleep study: this section enables the user to enter key measures from the polysomnogram such as apnoea–hypopnoea index (AHI), oxygen desaturation index (ODI), oxygen saturation and periodic limb movement index. The software interprets the results of the sleep study according to American Academy of Sleep Medicine criteria [24].
The software algorithms include programmed safety alerts throughout each section so that conditions mimicking the presentation of OSAS, such as hypoventilation, central sleep apnoea and neuromuscular disease will be highlighted. There are prompts for the user to act on the alerts generated (e.g. presence of driving-related drowsiness, significant hypoxia, critical hypertension, hypoventilation).
Following completion of the history and examination sections, the CGC processes these findings using an algorithmic approach and offers the user a pre-investigation diagnosis and suggests appropriate investigation (e.g. whether the findings merit prompting the operator to proceed to a sleep study based on this pre-investigation diagnosis). After the patient has had a sleep study, the operator is invited to enter the key results of the study and the CGC algorithms interpret those results integrating with the history and examination component to make a suggested final diagnosis. As an example, the software recognises the presence of symptoms and polysomnography findings that may point to coexisting or alternative diagnoses such as restless legs syndrome/periodic limb movement syndrome and also incorporates this into the patient report thus alerting the clinician.
The summary generated from the consultation becomes a standardised electronic record and, prompts clinicians towards interventions based on UK National Institute for Clinical Excellence (NICE) guidelines [25]. While providing these prompts the software itself does not make decisions, rather it suggests appropriate investigative or therapeutic interventions and leaves the user open to make the final choice as appropriate. At the end of the consultation a summary and management plan are created, which can be added to the patient records, printed and disseminated or e-mailed.
Stage 1: validation of the guided consultation when used by a by a sleep specialist (figure 1)
Consecutive subjects referred into the regional sleep service with suspected OSAS had a sleep study (ApneaLinkTM; Resmed) following receipt of the referral and before the outpatient appointment with the sleep specialist physician. At the outpatient visit, the sleep specialist physician, (BC) used the CGC to collect the history and clinical examination components only, but without access to either the results of the sleep study or any assessments/diagnoses made by the CGC. Following completion of the CGC data collection, the sleep specialist closed the CGC, reviewed the results of the sleep study and completed the remainder of the consultation without using the CGC.
Implementation of guided consultation when used by a specialist physician (Stage 1).
Separately, following this consultation, an independent researcher (Consultant Physician, RA), blind to the diagnosis reached by the sleep specialist in the outpatient clinic, entered the sleep study results into the CGC thus enabling the CGC to integrate the data collected from the history, examination and sleep study and to thus arrive at a final diagnosis. The final diagnosis reached by the CGC was reviewed by this independent researcher (RA) and compared to the final diagnosis reached in the outpatient clinic by the sleep specialist (BC) in the service.
Stage 2: validation of the guided consultation when used by a non-specialist (figure 2)
A second cohort of subjects referred consecutively to the regional sleep service with suspected OSAS were evaluated using the CGC by a nonmedical graduate (NLB) who was also a trained researcher with no previous sleep medicine training. The evaluation was done immediately following receipt of the referral and followed within 24 h by a sleep study (ApneaLinkTM; Resmed). The nonmedical graduate collected both history and examination components using the CGC, and subsequently inputted the results of the sleep study into the CGC after this was performed, which then formulated a suggested final diagnosis and suggested a management plan. Following this process, each patient was reviewed in the outpatient clinic by a sleep specialist physician blinded to the interpretations made by the CGC. For each case, the final diagnosis reached by the CGC when used by the nonmedical graduate was compared to the diagnosis arrived at by the sleep specialist physician.
Implementation of guided consultation when used by a nonspecialist (Stage 2).
In our sleep service, all sleep studies are manually inspected by a senior sleep physiologist. Apnoeas and hypopnoeas were scored manually according to American Academy of Sleep Medicine criteria [24]. Statistical analysis was performed using SPSS 26.0. Data are presented as mean±sd unless otherwise stated. Statistical significance was defined as a p-value <0.05. We used the independent sample t-test to identify significant differences in continuous variables and the Chi-squared test for categorical variables. During each stage of this evaluation, the strength of agreement between the final diagnosis reached by the CGC and that reached by the sleep specialist was calculated using the κ statistic according to the system proposed by Landis and Koch [26] (value of 0–0.2: slight agreement; 0.21–0.40: fair agreement; 0.41–0.6: moderate agreement; 0.61–0.80: substantial agreement and 0.81–1: almost perfect agreement).
Formal ethical approval to conduct the study was obtained from the Research Ethics Committee (REC) and the Health Research Authority (reference 19/HRA/6433)
Results
The key demographics of the study populations described in the two-step implementation process are outlined in table 1. No statistically significant differences were observed in baseline variables between the two cohorts (stage 1 and stage 2).
Study population demographics
Stage 1: comparison of guided consultation when used by a by physician
Overall, 100 subjects referred with suspected OSA/OSAS had data collected by a physician using the CGC, but without access to the CGC conclusions. No consultation took longer than 15 min using the CGC. The CGC correctly identified additional pathology in 17 subjects (8 with background hypertension, 7 with type 2 diabetes and 2 with ischaemic heart disease). Comparing the physician-collected CGC process with that of the sleep specialist's own diagnosis, both reached a final diagnosis of OSAS in 88 (88%) of cases (61% male, mean age 52±12 years; BMI 37±10 kg·m-2; ESS 10±6; AHI 30±26) and in 7 cases both concurred that there was no evidence of OSAS. In the remaining five cases, the sleep specialist felt that a diagnosis of OSAS did not explain the patient's presentation (transient nocturnal cough, nasal obstruction, tiredness due to sleep hygiene issues, periodic limb movement syndrome and unexplained tiredness), whereas the CGC final diagnosis was of possible OSAS but in all these 5 cases prompted a specialist referral to investigate the unexplained symptoms detected by the algorithms (table 2). Thus, in this cohort, the final diagnosis made by the CGC was concordant with that of the sleep specialist in 95% of cases and made a safe recommendation in the remainder. The κ value measuring agreement between the CGC and the sleep specialist for this stage was 0.59 (p<0.05).
Outcomes of the consultation when performed by the sleep specialist (n=100 cases)
Stage 2: comparison of the guided consultation used by a nonspecialist with a traditional specialist consultation
Overall, 100 subjects were evaluated with the CGC by the nonmedical graduate. The CGC identified that 35 subjects had background disorders (20 with hypertension, 6 with maturity-onset diabetes, 9 with chronic pain syndrome treated with analgesics, including opiates and gabapentin). Of these 100 subjects, 5 underwent the sleep study but did not attend their subsequent sleep specialist appointment, leaving 95 subjects to be evaluated (see table 3). No consultation took longer than 15 min using the CGC. Both the sleep specialist and the CGC made a firm diagnosis of OSAS in 83 of 95 subjects (87% (mean age 52±12 years; ESS 11±5; BMI 37±9 kg·m-2; AHI 26±22) of the 95) and agreed that 5 cases had no evidence of OSAS. Of these, one patient was diagnosed with periodic limb movement disorder but still discharged due to the absence of any alarming symptoms. In the final seven cases (four men, mean age 41±11 years; ESS 9±4; BMI 26±4 kg·m-2; AHI 3±1), the sleep specialist felt that there was no evidence of OSAS and discharged the patients, whereas the CGC algorithms prompted a specialist review to investigate unexplained symptoms. Thus overall, the final diagnosis made by the CGC was concordant with the sleep specialist in 93% (88 of 95) of cases (table 3) and in the remainder where there was some uncertainty, the CGC prompted for a specialist review. The κ value measuring agreement between the CGC and the sleep specialist for this stage was 0.56 (p<0.05).
Outcomes of the consultation when performed by the nonspecialist (n=95 cases)
Discussion
There has recently been considerable interest in applying technologies such as CDSSs in order to improve efficiency of existing patient pathways whilst retaining the high quality of standardised guideline-delivered practice [27]. This is particularly applicable in the area of sleep medicine, where it is estimated 85% of OSAS cases in the UK remain undiagnosed [28]. Furthermore, health economic studies show that accurate diagnosis of sleep apnoea carries significant benefit, with annual NHS savings of £28 million if all the cases of moderate and severe OSA were diagnosed and managed [28]. Our data demonstrate that within a busy clinical sleep medicine service, it is possible for nonspecialists working under the clinical oversight of a sleep specialist physician to assess new referrals with suspected OSAS using a CGC quickly (each consultation lasting15 min duration or less) with the CGC maintaining a high level of diagnostic accuracy with a concordance of 93% when compared to a sleep specialist physician.
A potential concern of modifying any clinical pathway by replacing routine specialist physician assessment with nonspecialist physicians assessing patients using computer software lies in the issue of patient safety particularly where patients may present with atypical symptoms. A sleep service will commonly encounter disorders whose symptoms may masquerade as OSAS (e.g. disorders of hypoventilation, narcolepsy and central sleep apnoea) [29]. A misdiagnosis or delay in diagnosis of these conditions may result in significant patient harm. The CGC used in this study is designed to capture such atypical symptoms and alert the operator that there are unexplained features. Furthermore, the CGC requires that the operator cannot override such an alert and has to enter an action. It has been reported that if clinical staff are able to override such alerts, then outcomes suffer [30]. In our analysis, when the CGC was used by the nonspecialist staff member, there were seven cases where the CGC was not concordant with the specialist opinion but in each case, the CGC made the safe conclusion to refer for a specialist review and the software did not miss any cases of OSAS hence underpinning it's safety in routine clinical practice. However, it must also be stressed that in our clinical pathway, all consultations performed using the CGC received clinical oversight by a sleep specialist physician which also serves as an important safety net in routine practice.
The use of a CDSS in the setting of OSAS has been described previously, yet these studies have been largely questionnaire-based [31–33]. A US study of 91 subjects with a background of ischaemic heart disease (of whom 75% had OSA) found that the use of an electronic handheld CDSS carried a sensitivity of 98.5% and a specificity of 86.9%. This was compared to a commonly used screening tool, the Berlin questionnaire, which reported a sensitivity and sensitivity of 87% and 39% respectively [32]. However, none of these papers describe the integration of sleep study data into the functioning of the CDSS in terms of what was reported though this functionality had been available in one [31]. These previous studies also excluded a number of key comorbidities, such as neurological disease and opiate use, which contrasts to our system where no patients were excluded and the conclusions of the analysis of any sleep study is integrated with the history and clinical examination thus providing a more global assessment. In contrast, our study, evaluated consecutive, unselected referrals for suspected OSAS using the CGC reflecting a “real world” scenario.
Despite this, our analysis has certain limitations. Our referral population represent those patients typically referred into our service with a suspicion of OSAS and thus consisted of a cohort who had a high pre-test probability of OSAS. Such a population may carry a different pre-test probability of OSAS in comparison to a less symptomatic population referred for screening purposes (e.g. to detect sleep-disordered breathing in occupational health settings, such as in professional drivers). However, the software does screen for comorbidities which increase the likelihood of underlying OSA being present in an individual, thus prompting the user to have a lower threshold to undertake sleep studies in these situations. Furthermore, the CGC interprets the sleep study data so such patients may then be diagnosed by the software after the sleep study has been undertaken. Critics may cite the κ scores described in our analysis as showing only a modest level of agreement between the CGC and the sleep specialist. However, an explanation for this may lie in the high pre-test probability of the sample having OSA coupled with the relatively small sample size in question and clearly future studies involving the use of the software require larger numbers for validation as opposed to our current analysis aiming for proof of principle. However, from a patient safety perspective, it should also be stressed that the CGC did not recommend discharging any patient in either cohort that had unexplained symptoms in the absence of a diagnosis of OSAS but instead recommended further investigation in each case. Our study not involve performing a clinical examination in the subjects who were assessed by the nonspecialist although this features as a component of the CGC hence abnormal anatomical findings that may be indicative of underlying OSAS such as significant tonsillar enlargement and the presence of craniofacial abnormalities such as retrognathia were not accounted for. However, examination of the upper airway could be carried out by a trained physiologist or a physician following the diagnosis of OSAS and at the time of initiating CPAP and any anatomical concerns referred for specialist review. We did not perform a formal health economic impact subsequent to implementation of the CGC and such analysis is vital in any healthcare setting. However, the principle of transferring 90% of the throughput to nonspecialist physicians or primary care physicians working under the oversight of specialist physicians and doing so safely using a CGC is almost certainly a cost-effective change, although this warrants further study.
If the data here are accepted and further studies validate the safety and diagnostic accuracy of such systems, then the impact on a sleep service could be both positive and substantial. Given the significant waiting lists to access sleep services both in the UK and beyond coupled with a shortage of medical specialists, then an option to have over 90% of suspected OSAS cases assessed by nonspecialist physicians using a CGC with sleep specialists providing clinical oversight and supervision has immediate practical and economic advantages. While this happens in a measure already with specialist nurses and respiratory sleep physiologists; systems such as this may act as a robust vehicle to extend this. This could enable any healthcare system using a CGC to assess sleep and breathing disorders and correctly prescribe CPAP support or to reassure that it is not required or to indicate when onwards specialist review is needed when diagnostic uncertainty exists. Thus, implementation of technological solutions such as the CGC described here within sleep services would result in scarce sleep specialist physician time being directed to target cases where there is diagnostic uncertainty or complexity, those OSAS cases with a suboptimal response to first-line therapy and cases where the CGC had flagged up key alerts or atypical symptoms. Similarly, healthcare commissioners can be assured that the standardised assessment was comprehensive and does not generate false positives leading to the over prescribing of CPAP therapy. Furthermore, as large volumes of patients come through this intelligent system, we suggest that machine-based learning should be applied to further refine decision prompts [34]. We have described the use for initial assessment, but having collected and stored the initial data, the next opportunity is to make use of informatics for objective monitoring and follow-up of their progress, and much of that may be possible remotely.
In conclusion, we have shown that an intelligent CGC system may be used both by a specialist physician or a nonspecialist working under specialist supervision to assess patients with suspected OSAS to a point of suggesting management in line with national guideline standards. A high level of diagnostic concordance between the CGC and a sleep specialist was observed with no adverse decisions. Using “evolved” clinical decision support systems such as a CGC can facilitate assessment of people with suspected OSA allowing new management pathways to be developed.
Acknowledgement
The CGC is owned by LungHealth Ltd. (Swaffham, UK).
Footnotes
Author contributions: The CGC was designed by B. Chakrabarti, R. Angus, M. Pearson, S. Craig, L. Davies, K. Sheridan and E. McKnight. P. England programmed the CGC. Data collection was performed by N. Lewis-Burke and B. Chakrabarti. All authors contributed to the study design, data analysis and manuscript preparation
Conflict of interest: B. Chakrabarti reports that the CGC is owned by LungHealth Ltd and has been previously employed for consultancy work with Lunghealth Ltd. (Swaffham, UK).
Conflict of interest: N. Lewis-Burke has nothing to disclose.
Conflict of interest: M. Pearson is a Director of LungHealth Ltd.
Conflict of interest: S. Craig has nothing to disclose.
Conflict of interest: L. Davies is a Director of Lunghealth Ltd.
Conflict of interest: K. Sheridan has nothing to disclose.
Conflict of interest: P. England is employed by LungHealth Ltd.
Conflict of interest: E. McKnight is a Director of LungHealth Ltd and reports that LungHealth Ltd. (Swaffham, UK) funded the salary of the software developer.
Conflict of interest: R. Angus is a Director of LungHealth Ltd.
- Received January 7, 2020.
- Accepted May 8, 2020.
- Copyright ©ERS 2020
This article is open access and distributed under the terms of the Creative Commons Attribution Non-Commercial Licence 4.0.