Abstract
Background Breathe is a mobile health (mHealth) application developed for the self-management of asthma in adults. There is evidence to suggest that mHealth interventions can be used for asthma control; however, their effects on the use of health services remain poorly understood. We sought to determine whether Breathe reduces health services use amongst asthma patients who used the app compared to controls who did not.
Methods The impact of Breathe on health services use was estimated using a quasi-experimental approach. Two groups of subjects who had participated in a previous randomised clinical trial were included: an intervention group of asthma patients who used the app for 12 months, and a group of controls who did not use the app but received equivalent quality asthma care. A third, external control group of asthma patients were matched to the intervention participants. Generalised linear mixed models were used to determine relative changes in rates of asthma hospitalisations, emergency department (ED) visits, outpatient physician visits and completion of pulmonary function tests (PFTs) over time.
Results A total of 677 individuals with asthma were included in the study: 132 in the intervention group, and 149 and 396 in the internal and external control groups, respectively. There were no statistically significant differences in the change of asthma hospitalisations, ED visits, physician office visits or completion of PFTs between the intervention group and either control group.
Conclusions Use of the Breathe app is not associated with changes in health services use in adults with asthma.
Abstract
Use of a mobile health application designed for asthma self-management was not associated with changes in asthma health services use or completion of pulmonary function testing in adults with asthma who use the application compared to those who did not https://bit.ly/2YojkeE
Introduction
Asthma is a respiratory illness affecting over 300 million people worldwide [1] and lifelong disease management is required to reduce the burden of asthma on patients [1, 2]. Comprehensive asthma management programmes rely on the integration of health services with the day-to-day control of symptoms and triggers by individual patients. Self-management programmes, which typically incorporate written asthma action plans, symptom monitoring and patient education, help to improve quality of life and decrease the rate of unplanned health care encounters [3, 4]. The importance of self-management is widely recognised and is a key component of national and international asthma guidelines [5–7].
The role of mobile health (mHealth) technologies is increasingly recognised in asthma self-management programmes. mHealth interventions, including those delivered over the Internet and smartphone or tablet computer applications (apps), are inexpensive, accessible alternatives that have the potential to reach a wide variety of patients [8]. The shift toward technology-supported disease management is demonstrated by the surge of asthma-specific apps that have been developed over the past two decades [9]. Considering challenges with the implementation and uptake of traditional paper-based programmes [8], health professionals are beginning to recommend the use of these apps in primary care [9].
Despite the abundance of available mHealth technologies, evidence needed to support their routine use in asthma care remains limited. A few randomised controlled trials (RCTs) [10–12], and one meta-review [13], have shown that digital interventions improve asthma knowledge and activity limitation in diverse asthma populations, as well as symptoms, medication use and quality of life in some instances [10, 11, 13]. However, existing studies that have examined the effect of apps on health services use (HSU) for asthma have found mixed results. Two RCTs [10, 14] demonstrated significant relative reductions in emergency department (ED) visits and hospitalisations, respectively, whereas others have found no differences between intervention and control subjects on ED visits, hospitalisations or primary care visits [13]. Further, it is unclear whether these results translate to “real-world” practice, suggesting a need to evaluate the effects of asthma apps as part of multicomponent care plans [8, 15].
The aim of the present study was to determine the effect of an asthma mHealth and web-based application named Breathe on patients’ HSU. Population-based health administrative data were used to tease out the effects of Breathe by comparing the intervention group to the control group from an RCT and to external, matched population-controls.
Methods
Study design and population
The mobile application Breathe was previously evaluated in an RCT, details of which have been described elsewhere [16]. In brief, Breathe [17] is an mHealth and web-based application developed for the collaborative self-management of asthma in adults. Breathe was designed to adhere to the Canadian Thoracic Society's principles of asthma self-management and was developed using a user-centred methodology. Breathe incorporates numerous features designed to provide patients with the knowledge and skills needed to control their condition, including: risk-reduction messaging, the display of environmental conditions, daily and weekly symptom assessment, and real-time access to a dynamic and personalised asthma action plan [18].
The present study compared outcomes in patients recruited in the previous Breathe RCT and matched controls generated from health administrative data in Ontario, Canada. The impact of Breathe on HSU was estimated using a quasi-experimental difference-in-difference (DID) approach that compares the changes in outcomes over time (e.g. before and after the intervention and between the intervention and control groups). In this study, the change in asthma HSU was compared between three study groups: 1) the Breathe intervention group (hereafter referred to as the Breathe group); 2) the internal control group that did not use Breathe (non-Breathe group); and 3) the propensity score-matched population controls identified from health administrative data (population control group). This study received ethics approval from the Hospital for Sick Children in Toronto, Ontario, Canada.
Intervention group (Breathe)
The study intervention group included individuals who had participated in the original RCT and were randomised to use Breathe. Participants in the Breathe group used the app for 12 months following enrolment, in addition to receiving continued usual care through an asthma clinic. Details on the development and specifications of Breathe have been published elsewhere [16, 18]. Inclusion criteria for participation in the Breathe RCT were: age 18 years or above at enrolment, a previous physician diagnosis of asthma, familiarity with and access to a mobile smartphone, laptop or desktop computer and English proficiency. Individuals were excluded if they had received a diagnosis for an additional chronic lung disease, such as cystic fibrosis, chronic obstructive lung disease or bronchiectasis.
Screening for individuals meeting the RCT inclusion criteria was completed by physicians during routine appointments or by certified respiratory educators through electronic chart review at Primary Care Asthma Program (PCAP) and PCAP-affiliated clinics. The PCAP is an evidence-based, high-quality asthma program administered in select clinics across the province of Ontario. Eligible PCAP patients were mailed invitations to participate in the RCT and enrolled between November 10, 2012 and April 22, 2014.
Internal control group (non-Breathe)
Study participants included in the internal control group were those who had been randomised to receive usual care in the original Breathe RCT. As were participants in the Breathe group, these individuals were recruited from PCAP and PCAP-affiliated clinics between November 10, 2012 and April 22, 2014 and were enrolled if they met the inclusion and exclusion criteria described above.
External control group (population control)
The external control group included asthma patients who were nonparticipants in the Breathe RCT. This group was formed by matching individuals in the Breathe group to individuals in the Ontario Asthma Surveillance Information System (OASIS), which identifies and follows Ontario residents who meet a validated case definition for asthma [19, 20]. Consistently with participants of the Breathe RCT, population controls had to be 18 years or older and diagnosed with asthma, and were excluded if they had been diagnosed with an additional chronic lung disease. To ensure the mutual exclusivity between the two control groups, any individual in the population control group who had ever been seen at a PCAP or PCAP-affiliated clinic was excluded.
Generating matched population controls to the Breathe group participants was completed using propensity scores. A propensity score was estimated using a logistic regression model, in which intervention treatment status was regressed on the variables of age, sex, asthma duration, socioeconomic status (SES), rural residency and overall medical comorbidity, all measured at the time of trial enrolment. Breathe group subjects were matched to controls on a 1:3 ratio on the logit of the propensity score (±0.2×sd), sex and age (±2 years). Successfully matched subjects in the population control group were assigned the same enrolment date as their intervention pair.
Data definitions and sources
This study used Ontario health administrative data housed at ICES (formerly known as the Institute for Clinical Evaluative Sciences). Individual-level data on HSU and sociodemographics were linked at ICES using unique encrypted identifiers. Primary outcomes of interest were asthma hospitalisations, ED visits, outpatient physician visits and completion of pulmonary function tests (PFTs) measured over 3 years: in the year prior to trial enrolment, during the intervention year and 1 year following trial completion. Additional details on the measurement of outcome variables and confounding variables included in the analyses are described in the online supplement.
Statistical analyses
Baseline characteristics for each study group were described using frequencies and proportions for categorical variables and means and sd for continuous variables. Comparisons between the Breathe group and each of the control groups were made using standardised differences. We considered a standardised difference of <0.1 (10%) to indicate balance of covariates [21, 22]. Inverse probability of treatment weighting (IPTW) using propensity scores was employed to control for any residual differences in observed baseline covariates between the Breathe and non-Breathe groups [23].
The DID approach was implemented by fitting generalised linear mixed models with a Poisson distribution for each of the outcomes of interest. Separate analyses were completed to compare the Breathe group to the control groups over three time periods: 1) from the 1 year prior to trial enrolment to the end of the trial (1 year previously); 2) from 1 year prior to trial enrolment to the 1 year following the end of trial (1 year post); and 3) from the year of the trial to 1 year post. All models were adjusted for important confounding variables (see online supplement) and the difference in outcomes was expressed as relative risk ratios (RRRs). All analyses were conducted using SAS Enterprise Guide 7.1 (SAS Institute Inc., Cary, NC, USA).
Sensitivity analyses
As the intervention and control participants in the Breathe RCT were both recruited from PCAP and PCAP-affiliated clinics with similar approaches and levels of quality of asthma care, we combined the Breathe and non-Breathe groups to create an enhanced care group. Using the same matching algorithm described previously, those in the enhanced care group were paired with population controls from the OASIS database. The balance of baseline covariates between these groups was assessed using standardised differences, and separate generalised linear mixed models were fitted for asthma hospitalisations, ED visits, physician visits and completion of PFT. To estimate the overall contribution of enhanced care on asthma HSU, RRRs were estimated for the enhanced care group relative to the population control group.
Results
A total of 300 individuals completed the Breathe RCT and consented to have their data linked for the purpose of this study. Of these, 19 could not be identified in the OASIS database and were excluded from our analyses. This left a total of 281 trial subjects, of whom 132 were in the Breathe group and 149 were in the non-Breathe group. All 132 Breathe subjects were successfully matched to 3 external population controls from OASIS (n=396), giving a total sample size of 677 asthma patients.
On average, study participants were 46 years old when they were enrolled in the study and had been living with asthma for 12 years. Most individuals were female (72%) and living in an urban area. Subjects across study arms had an average of five collapsed aggregated diagnostic groups (CADGs). Compared to non-Breathe subjects, those in the Breathe group were younger (45 versus 48 years), and a greater proportion were in the lower quintile(s) for several dimensions of the Ontario Marginalisation Index (table 1). Full balance of baseline covariates between the Breathe and non-Breathe groups was achieved following the application of IPTW, as indicated by a comparison of means and proportions using the weighted standardised difference [23].
Baseline characteristics of study sample by intervention group
Change in health services use
Table 2 shows the annual rates of HSU and completion of PFTs in each study group over the three time periods and figure 1 shows the trend in quarterly rates. With the exception of asthma hospitalisations, there was a consistent decrease in the rate of ED visits, physician office visits and PFTs over time in all groups. These trends were observed both during the study intervention year and 1 year post. Observed decreases were greatest in the Breathe group relative to the non-Breathe group and population controls at each of these time points. However, estimated from the DID models, these differences were not statistically significant (table 3).
Annual rates of health services utilisation and pulmonary function test completion by study group
Quarterly rates of health services use and pulmonary function testing over time and by intervention group.
DID models estimating relative changes in health services utilisation and PFT completion
Results of the sensitivity analyses showed no statistically significant differences in the change of asthma hospitalisation, ED visit or physician office visit rates between the enhanced care group and population controls. Compared to the population control group, the enhanced care group showed a greater change in the rate of PFT completion during the study year relative to 1 year prior. The rate in the enhanced care group decreased from 1.31 to 1.02 per person-year, whereas the rate in population controls increased slightly from 0.38 to 0.43 per person-year. Estimated from the DID model, this difference corresponds to an adjusted RRR of 1.21 (95% CI: 1.03, 1.42; p=0.024) (supplementary table E1).
Discussion
We conducted a three-group study including intervention and control participants from the Breathe RCT and matched population controls from health administrative data to estimate the effects of an asthma self-management app on HSU. We found no significant differences in the change in rates of asthma hospitalisations, ED visits, physician office visits or PFT completion in those who had used the Breathe app relative to internal controls or population controls who were not seen in enhanced care clinics and had not used the Breathe app.
Our findings are in line with those of the original Breathe RCT, which showed no significant differences in the change of quality of life between Breathe and non-Breathe subjects [16, 17]. The majority of studies examining the effects of mHealth interventions on asthma HSU have also reported no differences between intervention and control groups [13]. However, Liu and colleagues [14] demonstrated a significantly greater reduction in ED visits after 6 months in those who had used mobile-based self-care software compared to those receiving standard care. Similarly, Joseph et al. [10] found a significant reduction in the number of asthma hospitalisations among high-school students who used a web-based program relative to controls after 12 months. Discrepancies between these results and ours might be attributable to differences in the included participants; the former was restricted to patients with moderate-to-severe persistent asthma recruited from a tertiary care centre [14], and the latter focused on African American adolescents with active asthma [10]. It is possible that these participants were more amendable to the effects of asthma interventions and may have differed from those included in our study, who were not excluded on the basis of their asthma severity and represented a more general asthma population. Further, differences in the intervention features may preclude a direct comparison of their effects on asthma outcomes.
Although PFTs (e.g. spirometry) are recommended for asthma diagnosis and monitoring [6], our results showed a decrease in rates of PFT completion over time. The observed paradoxical decrease in PFT completion, especially in the enhanced care group, may be related to practices of primary care physicians at PCAP and PCAP-affiliated clinics, who do not routinely bill to the provincial public health insurance plan for this testing. As such, spirometry captured through administrative data may underestimate the true completion rate.
Observed outcome rates were generally lower for the population controls relative to the Breathe and non-Breathe groups. Although these differences were not statistically significant, there may be inherent differences between groups that may in part explain these differences. Notably, asthma patients who received treatment at a PCAP or PCAP-affiliated clinic may have been more severe asthma cases and, although propensity scores were used to achieve balance of covariates, residual confounding may exist.
Implications and future research
The results of our study do not support the use of mHealth interventions for reducing asthma HSU. Nonetheless, the continued use of Internet and mobile technologies for asthma self-management may still be warranted on the basis of their effects on other health-related outcomes. For example, a recent meta-review by Morrison and colleagues [13] found that, among the included clinical trials, all that examined knowledge and activity limitation showed improvements in these outcomes among intervention subjects relative to controls. In addition, a majority showed improvements in markers of self-care, quality of life and medication use, with a limited number also supporting the use of digital interventions for reductions in symptoms and school absenteeism [13]. Furthermore, although we did not find statistically significant differences based on DID results, we did observe absolute differences between the three groups, most notably for asthma hospitalisations. Given these differences, the impact of hospitalisation on patients, and the complexity of patient mHealth interaction, we suggest that future research focus on identifying target populations and developing mHealth functionality that supports the prevention of exacerbations that lead to hospitalisation.
mHealth self-management may be most effective if targeted to specific patient users. Whereas patients in our study were all 18 years or older (average 46 years), had asthma for an average of 12 years, and were not excluded on the basis of asthma severity, some research has suggested that these interventions might be most beneficial for patients with recently diagnosed asthma [15, 24], those with severe or poorly controlled disease, as well as younger persons who may have a stronger preference for technology-based self-management [11]. Indeed, Joseph et al. [10], who demonstrated positive effects of an online asthma intervention on hospitalisations, used a tailored messaging approach that was based on students’ responses to a lung health questionnaire. As highlighted by Andrews et al. [3] further qualitative and mixed-methods research will help to uncover the key barriers and facilitators to digital self-management, and will help to guide the development of more tailored interventions.
Strengths and limitations
Strengths of this study include its design and objective measurement of health services outcomes. We were able to capitalise on the rigorous methodology of a clinical trial, while also leveraging the use of health administrative data to account for variations in clinical practice. Using clinical trial data as well as propensity score-matching allowed us to create balanced study groups that were similar in terms of their baseline distribution of key covariates, such as age, sex, residence, SES and comorbidity. Second, we were able to objectively quantify rates of asthma hospitalisations, ED visits and physician office visits by measuring these outcomes through routinely collected, population-level health administrative data.
Interpretation of our findings should be considered in light of limitations. First, due to the unavailability of patient-level data, we were unable to examine potentially important outcome variables, such as changes in an exhaled nitric oxide test (e.g. exhaled nitric oxide fraction) or Asthma Control Test score. We were also unable to include these or other measures of asthma severity or symptom control in the algorithms used to match Breathe subjects to population controls, and residual confounding may partially explain our findings. Second, this study included adults aged 18 years or older and results may not be generalisable to children who, as previously mentioned, may have less experience with asthma self-management as well as a stronger preference for mHealth interventions. Finally, use of the Breathe app over time by participants in this group was inconsistent, and our study may have been underpowered to detect statistically significant differences between the Breathe group and controls using DID estimation.
Conclusions
There is little evidence to support the widespread use of Breathe, a novel mHealth intervention, to reduce rates of asthma hospitalisations, ED visits or physician office visits or increase uptake of PFT. Healthcare professionals who care for people with asthma are encouraged to consider the unique needs of their patients; asthma mHealth interventions may be beneficial for some patient subgroups, and may have greater effects on more proximal health outcomes. Additional participatory and qualitative research will help to uncover which features of mHealth interventions can be tailored to improve their applicability in the future.
Supplementary material
Supplementary Material
Please note: supplementary material is not edited by the Editorial Office, and is uploaded as it has been supplied by the author.
Supplementary material 00340-2019.supplement
Footnotes
This article has supplementary material available from openres.ersjournals.com.
Conflict of interest: T. To has nothing to disclose.
Conflict of interest: M.D. Lougheed reports an Ontario Lung Association/Canada Health Infoway sub-grant during the conduct of the study; other support from the AstraZeneca PRECISION Program, grants from Astra Zeneca, GlaxoSmithKline, Hoffman LaRoche Ltd. and Novartis, grants and other support from the Ontario Lung Association, grants from the Government of Ontario's Innovation Fund, Allergen NCE, Janssen and the Canadian Institutes of Health Research, other support from the Canadian Thoracic Society, and grants from the Manitoba Workers Compensation Board, outside the submitted work; and she is the nominated Canadian Thoracic Society representative on the Canadian Lung Association's Board of Directors.
Conflict of interest: R. McGihon has nothing to disclose.
Conflict of interest: J. Zhu has nothing to disclose.
Conflict of interest: S. Gupta has nothing to disclose.
Conflict of interest: C. Licskai reports grants from The Lung Association Ontario during the conduct of the study.
Support statement: This study was supported by ICES, which is funded by an annual grant from the Ontario Ministry of Health and Long-Term Care. The Ontario Lung Association through investment from the Canada Health Infoway's Consumer Health Solutions Program and the Ministry of Health and Long-term Care (Asthma Program) supported the development of Breathe. Parts of this material are based on data and information compiled and provided by the Canadian Institute for Health Information. The analyses, conclusions, opinions and statements expressed herein are solely those of the authors and do not reflect those of the funding or data sources; no endorsement is intended or should be inferred. Funding information for this article has been deposited with the Crossref Funder Registry.
- Received December 4, 2019.
- Accepted June 3, 2020.
- Copyright ©ERS 2020
This article is open access and distributed under the terms of the Creative Commons Attribution Non-Commercial Licence 4.0.
References
