Abstract
Introduction Objective cough frequency is a key clinical end-point but existing wearable monitors are limited to 24-h recordings. Albus Home uses contactless motion, acoustic and environmental sensors to monitor multiple metrics, including respiratory rate and cough without encroaching on patient lifestyle. The aim of this study was to evaluate measurement characteristics of nocturnal cough monitoring by Albus Home compared to manual counts.
Methods Adults with respiratory conditions underwent overnight monitoring using Albus Home in their usual bedroom environments. Participants set-up the plug-and-play device themselves. For reference counts, each audio recording was counted by two annotators, and cough defined as explosive phases audio-visually labelled by both. In parallel, recordings were processed by a proprietary Albus system, comprising a deep-learning algorithm with a human screening step for verifying or excluding occasional events that mimic cough. Performance of the Albus system in detecting individual cough events and reporting hourly cough counts was compared against reference counts.
Results 30 nights from 10 subjects comprised 375 hours of recording. Mean±sd coughs per night were 90±76. Coughs per hour ranged from 0 to 129. Albus counts were accurate across hours with high and low cough frequencies, with median sensitivity, specificity, positive predictive value and negative predictive values of 94.8, 100.0, 99.1 and 100.0%, respectively. Agreement between Albus and reference was strong (intra-class correlation coefficient (ICC) 0.99; 95% CI 0.99–0.99; p<0.001) and equivalent to agreement between observers and reference counts (ICC 0.98 and 0.99, respectively).
Conclusions Albus Home provides a unique, contactless and accurate system for cough monitoring, enabling collection of high-quality and potentially clinically relevant longitudinal data.
Abstract
The Albus Home respiratory monitoring solution provides an accurate system for nocturnal cough monitoring validated against manual cough counts. This contactless device enables reliable nightly monitoring for as long as required without adding burden. https://bit.ly/3C9ErGQ
Introduction
Nocturnal symptoms are known to be clinically relevant across many different clinical diagnoses, including in several respiratory conditions [1, 2]. The presence of nocturnal symptoms, such as coughing and nocturnal awakenings, is an indication of worsening asthma control and is thus routinely included as an important clinical question in clinical guidelines and patient-reported outcome measures [3–6]. In asthma, chronobiological studies of circadian rhythms have also shown worsening lung function and airway inflammation during the night [7], which is associated with nocturnal and early morning coughing [8, 9]. Importantly, nocturnal cough correlates with daytime coughing and with worsening clinical states in COPD [10].
Nocturnal cough frequency has also been shown to improve after treatment in asthma [11]. This suggests that the measurement of nocturnal coughs is an important marker of disease activity which can alert the patient and clinicians to modify therapy or take pre-emptive action. This has the potential to improve patient engagement in their own clinical care and also provide an additional tool for clinicians which reflects objective symptoms beyond the brief encounters in primary and secondary care.
The usual method of assessing nocturnal symptoms in clinical care or research is by symptom diaries and patient-reported outcome measures (PROMs). These rely on patient/carer recall for the previous night, week or month [12]. However, under-reporting is common using these methods compared to objective methods [13]. In the case of cough, objective cough frequency monitoring has become a key end-point in clinical trials [14] in addition to validated PROMs such as the Leicester Cough Questionnaire [15].
Currently, there are two cough frequency monitors that have been validated, and both are systems that need to be worn by patients [16–18]. One of the limitations of existing wearable systems is that due to their cumbersome nature, they are not suitable for monitoring for more than a few days at a time, which translates to difficult deployment in clinical care settings [19]. In clinical trials, they are limited to 24-hour recordings at baseline and after pre-defined periods of time [20–22]. Moreover, the effect of temporarily wearing an intrusive cough monitor on cough frequency is unknown [14].
Albus Home RD (Research Device; Albus Health, Oxford, UK) is a CE-marked contactless and automated bedside device that enables nocturnal monitoring without the need to do or wear anything. Albus Home RD monitors multiple metrics including respiratory rate, cough and room air quality, with potential to capture additional metrics, such as body motion and sleep-related metrics [23, 24]. The contactless nature of this system enables collection of longitudinal data across several nights and weeks without adding burden. The aim of this present work was to evaluate the performance of the overnight cough monitoring component of the Albus Home system.
Methods
Albus Home system
The Albus Home system consists of the Albus Home RD bedside device that records raw data and proprietary algorithms that report clinical metrics. Albus Home RD is a multi-metric nocturnal monitoring device that employs motion, acoustic and environmental sensors designed for contactless long-term home monitoring. For the cough monitoring aspect evaluated in this present work, a free-field microphone captures full audio recordings. Study participants were either provided with the Albus Home RD at in-person visits or it was posted out to them after remote visits. All participants were monitored in their usual home bedroom environments (figure 1). The device is a plug-and-play device simply placed on the bedside, and all participants self set-up the device after watching a brief set-up video. For participants taking part through remote visits, the device placement was confirmed through a video-call. No behavioural restrictions were instructed, and participants were free to have other sources of noise such as television and music. The recording hours were set around individual sleeping schedules and preferences: for five participants the device was set to start around their usual sleeping and waking times, whilst the other five participants opted for a longer recording schedule starting from the late afternoon to allow for differing sleeping patterns. Completed recordings were stored on secure USB drives inserted in the device, which were returned to the study team at study visits or collected by courier. All data stayed encrypted during the recording and in transit and stored in secure password-protected servers.
Example use-case and placement of Albus Home RD.
Completed overnight audio recordings from Albus Home RD were analysed using a proprietary system, which comprises an automated deep-learning algorithm (v1.2) for detecting cough events and a human screening step to remove occasional events that mimic cough and to confirm the cough count. The Albus system detects individual cough events as well as reports cough frequency per hour for each hour of every recording.
Subjects in validation set
10 adult subjects with respiratory conditions were included from two NHS Research Ethics Committee (20/YH/0041, 19/SC/0417) approved observational studies which involved overnight monitoring using the Albus Home RD. Three nights of monitoring per subject were included in the analysis, totalling 30 nights of data. Crucially, the whole validation set was fully held-out, meaning that none of these 30 nights and no other data from these 10 subjects of the validation set were previously seen by the cough detection machine learning pipeline during training and development.
Reference cough counts and validation of the Albus system
To validate the performance of cough counts from the Albus system, raw audio recordings were independently counted by two trained human annotators, with audio-visual labelling of the explosive phase of coughs using Audacity [25, 26]. Manual counting of audio recordings has been used as the gold standard reference in previous validation studies of cough monitors [26]. However, prior work also reports inter-observer variability in cough counts, especially relating to ambiguous cough-like sounds, such as clearing of throat, or when cough sounds are quiet or distant [17]. Therefore, in this work, a gold standard reference cough was defined as when two annotators both labelled a cough, as previously suggested [16]. Similarly, reference cough counts per hour were defined as the number of coughs in each hour where both annotators had agreed.
Analysis
The performance of the Albus system in detecting individual coughs compared to reference counts was reported as sensitivity, specificity, positive predictive value (PPV, also known as precision) and negative predictive value (NPV). Additionally, cough frequency reported as coughs per hour was compared for the Albus system and reference counts and intra-class correlation coefficient (ICC) calculated [27]. For comparison, the ICC between hourly cough count results from respective annotators with the reference counts were also reported. Finally agreement between the Albus system and reference hourly counts was assessed using Bland–Altman analysis [28].
Results
Three overnight recordings from 10 adult subjects (five males) with chronic respiratory conditions were included in this validation. There was a wide range of hourly cough counts from 0 to 129 coughs·h−1. Summary characteristics of the validation set data are provided in table 1.
Validation set data
Performance of Albus cough monitoring system
The Albus system had a median (Q1 to Q3) participant sensitivity of 94.8 (89.7–97.7)%, specificity of 100.0 (100.0–100.0)%, PPV of 99.1 (97.1–99.5)% and NPV of 100.0 (100.0–100.0)% for detecting individual coughs compared to the reference. The mean length of events for human screening in the Albus system was <2 min per overnight recording (mean 12.5 h), equivalent to 0.2% of a recording. When cough frequency as counts per hour were compared between the Albus system and reference counts for all 375 h, the overall mean±sd difference was −0.6±2.0 coughs·h−1, and the ICC was 0.99 (95% CI 0.99–0.99; p<0.001; figure 2a). The agreement and ICC between the Albus system and reference coughs per hour was equivalent to that between human annotator coughs per hour and the reference counts (0.98 and 0.99 respectively). The Albus system also had strong agreement with the reference counts in Bland–Altman analysis of intra- and inter-subject performance when comparing cough counts for each hour (figure 2a) and mean hourly cough counts for each night (figure 2b). Figure 2b also shows the varying hourly and mean hourly cough frequencies by subject. The limits of agreement in the difference in hourly cough counts between the Albus system and reference was approximately ±4 coughs·h−1, and in nightly mean hourly cough counts ±2.5 coughs·h−1. Several subjects showed a similar mean hourly cough count across the three nights, but there were also subjects with significant differences between nights, for instance nightly mean coughs per hour ranging from 13 to 34 within the same subject (figure 2b).
Bland–Altman plots showing agreement in cough counts between Albus system and human reference counts. Solid lines show mean difference. Dotted lines show 95% limits of agreement (1.96 × sd). Data points are shown as different shapes for each condition (as per key) and in different colours for each subject. a) Comparison of hourly cough counts for each hour of validation set (n=375; multiple hours may overlap in count and difference). Mean difference: −0.6±2.0 coughs·h−1. b) Comparison of mean hourly cough counts for each night of validation set (n=30; three nights each for 10 subjects). Mean difference: −0.7±1.3 coughs·h−1.
Discussion
In this work, we have validated the cough monitoring performance of Albus Home, a contactless bedside multi-metric nocturnal monitoring system, compared against gold standard reference human counts. This study demonstrates that the cough monitoring component of the Albus system was accurate across hours with high and low cough frequencies, with a median participant sensitivity, specificity, PPV and NPV of 94.8, 100.0, 99.1 and 100.0% respectively for detecting individual coughs. When comparing cough counts per hour, there was close agreement between the Albus system and reference counts with an ICC of 0.99, with tight inter- and intra-subject performance. Prior work on validating the performance of cough detection proposed the method of comparing variability introduced by a system to the intrinsic variability between counters [17]. To this end, this present work showed the variability between the Albus system and reference counts was within the intrinsic variability of cough labelling by two human annotators.
This work has several strengths. Firstly, the validation set constituted three nights from 10 subjects with hourly cough frequencies ranging from 0 to 129 coughs·h−1comprising diverse spontaneous cough sounds, evaluating the system's performance at detecting both low and high counts per hour. Moreover, the analysis of multiple nights per subject allowed an assessment of both inter- and intra-subject performance. Secondly, every subject included in the validation set were fully held-out, such that the deep-learning model had not “seen” any data from any of these subjects. This robust validation methodology tests the performance of the system to generalise when tested on new patients. Thirdly, every hour of this 375-hour validation set was independently counted by two trained annotators. Both prior literature and this present work has found that some level of inter-observer variability does exist when two people label coughs. Although many coughs are typical acoustically and visually in waveform [25], with real-world patient data there are many acoustic sounds that are ambiguous, for instance leading to disagreement between annotators whether a sound is a cough or clearing of throat [17]. Therefore, validating a system against a single annotator may not constitute the most reliable gold standard, and in this work, though hugely resource-intensive, all recordings were double annotated and our gold standard set as sounds that were recognised as cough by both observers, as has been suggested before [16].
Another strength of this work was that all subjects (including older subjects with COPD) performed monitoring by self-setting up the monitoring device in their usual home bedroom environments. Moreover, participants were not instructed to modify any aspects of their usual behaviours, including playing of TVs, radios and music, which in this study was a common source of ambient noise before and around bedtime. Thus, this work replicated the use-case of the system in real-life deployment, and presented the accuracy results within this context. Many of the subjects in this work received the device by post, then set it up using a video resource or remote video-call with a researcher. The plug-and-play nature of this system minimises the barriers to enabling longitudinal data collection, especially with remote deployment more prominent since the COVID-19 pandemic.
A limitation of this work is that it was outside its scope to characterise cough frequencies across conditions or assess the magnitude of change in coughing that is clinically relevant. Correspondingly, the clinical state of each subject at the time of monitoring was not assessed, in part due to the remote nature of participation by most subjects. Regardless, these results suggest that significant variation in cough frequency is possible between nights within the same patient (figure 2b) that could be captured by a long-term contactless solution. Given that this present work is one of the first presentations of a minimally burdensome system for longitudinal nocturnal cough monitoring, there is sparse data on the magnitude of clinically meaningful differences. However, previous work showed that mean nocturnal hourly cough counts can vary by over 15 coughs·h−1 during recovery from exacerbations in COPD [10], significantly greater than the limits of agreement for this system. In the design of this study, it was deemed important that a cough detection system needed to perform accurately in patients with a high burden of cough, but also not to produce high numbers of false positives in hours with minimal or low cough counts. To this end, subjects did not need a diagnosis of chronic cough to participate, and the resultant validation set contains a mix of subjects exhibiting high counts and several with lower counts; consistent with findings of lower cough frequencies at night than the day, the majority of hours exhibited hourly frequencies <40. Thus, future work will seek to evaluate the system in a greater number of patients with higher burdens of cough, across a greater diversity of conditions, including refractory or unexplained chronic cough, as well as healthy control subjects.
Another limitation of this work is that by design it focused on night-time monitoring, with the operative range being the bedroom (though the system could technically be deployed in other rooms where patients spend substantial time). Though the system naturally captures the period when subjects are sleeping, the monitoring can also comprise the time before subjects fall asleep and just after they wake up, which are known to have higher counts than during sleeping hours and may correlate with daytime or 24-hour frequencies [8, 16]. As this present work did not include 24-hour monitoring, it is not possible to compare the magnitude of clinically discriminatory signals between daytime and night-time cough. As was illustrated by the results of the Sivopixant phase 2A trial [29] where there was a significant reduction in 24-hour counts but not in daytime counts, different cough frequency metrics may influence the result of trials. Exploring this was outside the scope of this study, but future work will seek to evaluate Albus Home overnight cough counts as a means of discriminating patients or changes within a patient, and compare this with 24-hour or daytime cough frequencies and pre-existing validated cough monitors. Regardless, prior work has found that nocturnal and daytime coughing are strongly correlated [10]. Moreover, though the absolute counts are significantly lower at night than the day [19], studies have indicated that nocturnal cough frequency is more strongly correlated to subjective measures than daytime [30]. Recent work, where 24-hour cough frequency was reported using the validated Leicester Cough Monitor software, found that night-time cough frequency was significantly higher in asthmatic patients than non-asthmatic patients, and that improvement in cough frequency in asthmatic patients upon treatment was greater during night-time than during the day [11]. A final potential limitation to consider is that co-occupants may confound a signal to a greater extent with the Albus system than with a wearable device. Previous studies have shown that cough frequency in normal subjects at night is very low [31], so we doubt this is a significant limitation. However, further studies are required to evaluate the impact of other bedroom occupants on the measurement characteristics of the device.
The main advantage of current wearable cough monitoring systems is that they allow both daytime and night-time monitoring. However, conversely the wearable nature means that recordings are usually confined to a limited number of 24-hour recordings. This limits the amount of data capture in clinical studies, whilst not being feasible to adopt in clinical practice [19]. Because patients can experience day-to-day variation in symptoms, 24-hour recordings may risk being confounded by normal variation that adds noise. Moreover, it has not been studied whether the intrusive activity of wearing a cough monitor for a short 24-hour period has any behavioural or psychological impact on study participants, for instance in reducing or increasing coughs, or avoidance of behaviours that affect coughing like smoking [32]. Ultimately, the goal of monitoring tools is to capture clinically meaningful differences over time. The authors carefully hypothesise that though limited to night-time, longitudinal monitoring of nocturnal signs over continuous periods of several days or weeks could have better discriminatory power than 24-hour recordings confined to one-off days, whilst being less impacted by temporary behavioural changes or normal variability. This would the subject of future study.
Finally, in considering the potential clinical utility of the Albus Home bedside system, it is important to note that nocturnal cough is one of several metrics that it can monitor. The device utilises multiple sensors, including motion, acoustic and environmental, and this raw monitoring data enables the extraction of a breadth of potentially informative metrics. This includes respiratory rate (RR), coughing, room air quality, body motion and sleep-related metrics. For instance, RR is one of the key vital signs to change in several clinical deteriorations [33, 34], but for which there are limited options for continuous and accurate monitoring in the home. Likewise, evidence for the clinical relevance of air quality and pollution in respiratory conditions is growing [35, 36]. As a multi-metric nocturnal monitoring device, the system has applicability in the context of clinical research to enable longitudinal data collection without adding participant burden. Given the contactless yet accurate nature of monitoring, the system also has potential for use in clinical care applications, such as the early predictions of exacerbations and monitoring the response to treatments. Further work would need to investigate the clinical signals for such use-cases.
Conclusions
In summary, the Albus Home system provides a novel contactless and accurate system for nocturnal monitoring of cough. The performance of the system was validated in an unseen and representative validation set of subjects with respiratory conditions monitored in real-world conditions. With the additional potential to concurrently capture a variety of clinically informative nocturnal metrics, the Albus system enables collection of high quantity and quality of longitudinal data in clinical trials without adding patient or researcher burden.
Footnotes
Provenance: Submitted article, peer reviewed.
Ethical approval: This work has been conducted in accordance with the principles of the World Medical Association Declaration of Helsinki. Study protocols received independent NHS research ethics committee approval.
Informed consent: All subjected provided written informed consent.
Author contributions: W. Do, R. Russell, C. Wheeler, M. De Vos, M. Bafadhel and I. Pavord contributed to the conception and design of the work. W. Do, C. Wheeler, G. Cunningham and V. Khanna contributed to the acquisition of data. H. Javed, C. Dogan and M. De Vos contributed to system development. W. Do, R. Russell, H. Javed, C. Dogan, M. De Vos, I. Satia, M. Bafadhel and I. Pavord contributed to the analysis or interpretation of data. All authors contributed to the critical revision of the manuscript and gave approval for submission.
Conflict of interest: W. Do is a co-founding shareholder and director of Albus Health. R. Russell is a scientific advisor and minority shareholder of Albus Health; and reports grants from AstraZeneca, and personal fees from Boehringer Ingelheim, Chiesi UK and GlaxoSmithKline, outside the submitted work. C. Wheeler, H. Javed, C. Dogan, G. Cunningham and V. Khanna were employees of Albus Health during the conduct of the study. M. De Vos reports consulting fees and minority shareholding from Albus Health; and reports grants from J&J/Janssen and public government outside the submitted work. I. Satia reports grants from Merck, GSK, Bellus and Bayer, personal speaker and consulting fees from GSK, AstraZeneca, Merck, Genentech and Respiplus outside the submitted work, and is an associate editor of this journal. M. Bafadhel is a scientific advisor and minority shareholder of Albus Health, reports grants from AstraZeneca, personal fees from AstraZeneca, Chiesi, and GlaxoSmithKline, and is a scientific advisor of ProAxsis, outside of submitted work. I. Pavord has received speaker's honoraria for speaking at sponsored meetings from AstraZeneca, Boehringer Ingelheim, Aerocrine AB, Almirall, Novartis, Teva, Chiesi, Sanofi/Regeneron, Menarini and GSK, and payments for organising educational events from AstraZeneca, GSK, Sanofi/Regeneron, and Teva; has received honoraria for attending advisory panels with Genentech, Sanofi/Regeneron, AstraZeneca, Boehringer Ingelheim, GSK, Novartis, Teva, Merck, Circassia, Chiesi and Knopp, and payments to support FDA approval meetings from GSK; has received sponsorship to attend international scientific meetings from Boehringer Ingelheim, GSK, AstraZeneca, Teva and Chiesi; has received grants from Chiesi and Sanofi Genzyme; and is co-patent holder of the rights to the Leicester Cough Questionnaire and has received payments for its use in clinical trials from Merck, Bayer and Insmed, outside the submitted work.
Support statement: This work was funded by Albus Health (registered BreatheOx Limited). I. Satia is currently supported by the E.J. Moran Campbell Early Career Award, Department of Medicine, McMaster University. Funding information for this article has been deposited with the Crossref Funder Registry.
- Received June 1, 2022.
- Accepted August 8, 2022.
- Copyright ©The authors or their employers 2022
This version is distributed under the terms of the Creative Commons Attribution Non-Commercial Licence 4.0. For commercial reproduction rights and permissions contact permissions{at}ersnet.org
References
