Abstract
Background The Swedish Registry of Respiratory Failure (Swedevox) collects nationwide data on patients starting continuous positive airway pressure (CPAP) treatment, long-term mechanical ventilator (LTMV) and long-term oxygen therapy (LTOT). We validated key information in Swedevox against source data from medical records.
Methods This was a retrospective validation study of patients starting CPAP (n=175), LTMV (n=177) or LTOT (n=175) across seven centres 2013–2017. Agreement with medical record data was analysed using differences in means (sd) and proportion (%) of a selection of clinically relevant variables. Variables of interest included for CPAP: apnoea–hypopnoea index (AHI), height, weight, body mass index (BMI) and Epworth Sleepiness Scale (ESS) score; for LTMV: date of blood gas, arterial carbon dioxide tension (PaCO2) (breathing air), weight and diagnosis group; and for LTOT: blood gases breathing air and oxygen, spirometry and main diagnosis.
Results Data on CPAP and LTOT had very high validity across all evaluated variables (all <5% discrepancy). For LTMV, variability was higher against source information for PaCO2 (>0.5 kPa in 25.9%), weight (>5 kg in 47.5%) and diagnosis group. Inconsistency was higher for patients starting LTMV acutely versus electively (PaCO2 difference >0.5 kPa in 36% versus 21%, p<0.05, respectively). However, there were no signs of systematic bias (mean differences close to zero) across the evaluated variables.
Conclusion Validity of Swedevox data, compared with medical records, was very high for CPAP, LTMV and LTOT. The large sample size and lack of systematic differences support that Swedevox data are valid for healthcare quality assessment and research.
Abstract
The National Swedish Registry for Respiratory Failure (Swedevox) of CPAP, long-term mechanical ventilator and long-term oxygen has high validity compared with medical records, and is suitable for follow-up of medical care and for clinical research https://bit.ly/31ACMY6
Introduction
Continuous positive airway pressure (CPAP), long-term mechanical ventilation (LTMV) and long-term oxygen therapy (LTOT) are established treatments for obstructive sleep apnoea (OSA) [1], hypoventilation [2–4], and for chronic severe hypoxaemia [5], respectively.
CPAP treatment [1] is established for patients with verified sleep apnoea with excessive daytime sleepiness (strong recommendation), reduced sleep-related quality of life (conditional recommendation), or comorbid hypertension (conditional recommendation).
LTMV is primarily used for chronic extrapulmonary hypoventilation disorders, such as obesity hypoventilation and neurological diseases [2]. Its use for treatment of chronic hypercapnic respiratory failure in COPD has been controversial, but recent research seem to have established its role in selected patients [3].
LTOT improves survival in severe hypoxaemia defined by partial pressure of arterial oxygen (PaO2) <7.4 kPa breathing room air at rest, or <7.8 kPa together with signs of right heart failure or secondary polycythaemia [6, 7]. Evidence for benefit pertains to patients with hypoxaemia that is chronic, defined as persisting ≥3 weeks despite optimal treatment for the underlying disease(s).
The Swedish National Registry of Respiratory Failure (Swedevox) collects nationwide data prospectively, on patients starting LTOT since 1987 (coverage about 85%) [8], LTMV since 1996 (estimated coverage 90%), and CPAP since 2010 (current coverage about 75%). Swedevox data are extensively used for assessing and informing about clinical practice across the country and in previous and ongoing research [9–16].
Validation of data entered into the Swedevox registry compared with medical records has not yet been performed. Such a validation is important to evaluate data quality, finding ways to improve correct entry and data capture, and most importantly to ensure scientific validity. The aim of this study was to validate registered data in Swedevox against medical records for a selection of key variables in the three treatment arms, CPAP, LTMV and LTOT.
Material and methods
Study design and population
This was a multicentre, retrospective validation study comparing registered Swedevox data with the patients' medical records.
Inclusion criteria were: patients registered in Swedevox between 2013 and 2017, with complete data on the evaluated validation variables (listed in the next section), who were registered at any of the centres participating in the validation (pulmonary departments at hospitals in Blekinge, Gothenburg, Gävle, Halmstad, Lund/Malmö, Stockholm (Solna), Luleå (Sunderbyn) and Uppsala). The centres were selected as they represent a substantial number of patients in the Swedevox registry due to a long period of reporting and also represent both larger and smaller regions across the entire country.
Of the patients in Swedevox during the time period at each centre, a random sample of 25 patients aged ≥18 years was selected for each treatment arm (CPAP, LTMV and LTOT). One study centre was later excluded (Solna) due to inability to provide data. In contrast, the centre Lund/Malmö included data from a larger randomised sample of LTMV (n=30), as Blekinge had only 22 eligible LTMV patients during the time period. The final validation population comprised 175 CPAP, 177 LTMV and 175 LTOT patients (table 1).
Characteristics by centre and treatment arm
Data entry in Swedevox
Data entered into the web-based Swedevox registry are obtained from medical records mainly by specialised nurses. Data should be obtained directly from the medical files, including the main diagnosis and specific sources such as questionnaires. Numerical data such as weight, sleep apnoea severity and blood gases should be representative for the situation when the decision was taken to start the treatment [8].
Current strategies to increase validity of registered Swedevox data include so-called hard entry limits, such as limits on the age span that can be entered, and also so-called soft validation checks that prompt the user when entering data outside the indications or physiologically unlikely (but not impossible) values, such as for blood gases.
Validation variables
A set of variables of interest was selected by the steering committee of the Swedevox registry in order to represent important markers for disease aetiology and severity, comorbidities and quality markers of patient management at the different centres. The following variables were validated: 1) For CPAP: start date, baseline apnoea–hypopnoea index (AHI), weight (kg), height (cm), body mass index ((BMI) kg·m−2), and Epworth Sleepiness Scale (ESS) score [17]; 2) for LTMV: start date, date of blood gas assessment breathing air (before starting therapy), PaCO2 (air), weight (kg), and main diagnosis group (amyotrophic lateral sclerosis (ALS), other neuromuscular disease (NMD), respiratory disease, obesity hypoventilation syndrome (OHS), thoracic restriction, or other); and 3) for LTOT: start date, PaO2 (air), PaO2 (oxygen), forced expired volume in 1 s (FEV1), vital capacity ((VC) defined as the highest value of the slow and forced VC), and the main diagnosis categorised as COPD, α-1-antitrypsin deficiency (AATD) with emphysema, other respiratory disease, pulmonary fibrosis (PF), sarcoidosis, other parenchymal disease, pulmonary arterial hypertension, chronic pulmonary embolism, other pulmonary vascular disease, heart disease, thoracic deformity, hypoventilation, tumour in lung or pleura, or other. The validation was restricted to diagnoses with more than five patients.
Procedures of the validation
A standardised data entry sheet was used, including the patients' Swedish personal identity number (used to link the data), treatment arm (CPAP, LTMV or LTOT), date of registration in Swedevox, and entry fields for each variable to be obtained from the medical records. Data were entered by co-authors and staff at each centre and were returned to and quality checked by the principal investigator (ME). The data were then compared with the corresponding data in the Swedevox registry. Cut-off limits for differences regarded as clinically significant were defined by consensus by the Swedevox steering board and are listed in tables 2–4. The cut-off for ESS was set to its published minimal clinically important difference of 2 points for that scale [17].
Continuous positive airway pressure data: agreement between Swedevox and medical records and proportion of patients with clinically significant differences
Statistical analyses
Baseline patient characteristics were summarised using mean with sd and median with range or interquartile range (IQR) for continuous variables with normal and skewed distribution, respectively. Categorical variables were expressed as frequencies and percentages. Representativeness of the validation sample for all patients in Swedevox was assessed by comparing the characteristics in Swedevox between patients included and not included in the validation sample during the study time period (2013 to 2017). Differences were compared using t-tests and Wilcoxon rank-sum tests for continuous variables with normal and skewed distributions, respectively and using Chi-squared tests for categorical variables.
Agreement between the Swedevox registry and medical record data was evaluated as mean differences with 95% confidence intervals, and the prevalence of differences above a pre-specified cut-off for each variable. Statistical analyses were conducted using the software packages Stata, version 16.0 (StataCorp LP; College Station, TX, USA).
Ethical considerations
The study protocol was approved by the Head of Department as part of quality assessment of the registration process in Swedevox at each participating centre. According to Swedish law and research regulation, all participants were informed and had the opportunity to opt out from being registered in Swedevox, and individual consent for participation was waived.
Results
A total of 527 patients were included in the validation across the seven centres (table 1). Within each treatment arm, the age and sex distribution were similar across the centres (table 1). Characteristics of the validation sample were overall similar to patients in the Swedevox registry who were not included in the validation during the time period, for CPAP (supplemental table s1), LTMV (supplemental table s2) and LTOT (supplemental table s3).
CPAP
CPAP data validity was very high (table 2). The mean differences (Swedevox versus medical records) for the key variables start date, AHI, BMI and ESS were close to zero; and the proportion of differences being categorised as clinically significant was <5% for all comparisons between Swedevox registry and medical records.
LTMV
LTMV data validity is shown in table 3. There were no or very small systematic differences between the registry and medical records data for the key variables. There was higher variability than for CPAP: start date of LTMV differed by more than 7 days in 9.1% of cases, and similar time differences were seen for 18.6% of dates of blood gas assessments. The difference in PaCO2 between registry and source data exceeded >0.5 kPa in 25.9% of patients, and a weight difference of >5 kg was detected in 47.5% of cases. However, the differences were evenly distributed leading to a mean difference near zero, indicating no systematic bias. When analysed separately for acute and elective LTMV start, agreement was higher for patients starting LTMV in the elective setting (table 3). This difference was also seen between acute and elective starts when analysing by each underlying condition (table s4 in the supplement).
Long-term mechanical ventilation data: agreement between Swedevox and medical records and proportion of patients with clinically significant differences
Long-term oxygen therapy data: agreement between Swedevox and medical records and proportion of patients with clinically significant differences
Agreement for LTMV main diagnosis group in Swedevox compared with medical records was high for the main diagnosis groups of ALS (95%), respiratory disease (87%), other NMDs (80%) and slightly lower for OHS (73%), as shown in figure 1. Nineteen patients with OHS according to Swedevox had a discrepancy in diagnoses in the comparison; of these, 16 had a diagnosis of respiratory disease in the medical records (supplemental table s5).Vice versa, the majority (4 out of 7) of patients classified with respiratory disease in Swedevox and discrepancy were classified as OHS in medical records. For most misclassified patients, the diagnosis group based on medical records was not captured by the variable for the additional diagnosis group in Swedevox.
Agreement, for patients with long-term mechanical ventilation, of the main diagnosis group between Swedevox and medical records. ALS: amyotrophic lateral sclerosis; NMD: neuromuscular disease; OHS: obesity hypoventilation syndrome.
LTOT
In the LTOT arm of the registry, mean differences were very small and close to zero for all assessed variables (table 4). Variability was low for treatment start date and PaO2 (breathing room air), whereas the rate of substantial differences were higher for PaO2 (breathing oxygen) (14.4%) and VC (16.9%).
Agreement between Swedevox and medical records for LTOT main diagnosis (figure 2) was very high for the major categories COPD (94%) and PF (89%, with some of the other cases categorised as PH), as shown in supplemental table s6.
Agreement, for patients with long-term oxygen therapy, of the main diagnosis between Swedevox and medical records, shown for diagnoses with n>5. PF: pulmonary fibrosis; PAH: pulmonary arterial hypertension.
Discussion
This is the first validation of a representative patient sample from the national Swedevox registry against source data (medical records) for patients starting CPAP, LTMV and LTOT during a 5-year period. Data quality in the registry was very high for both CPAP and LTOT across all validated variables. For LTMV, agreement was high especially for patients starting the therapy electively. As could be expected, there were some discrepancies for patients starting LTMV acutely in the hospital setting. However, variability was evenly distributed with no mean difference in registered values compared with those in medical records, indicating that there was no systematic bias. This validation study strongly supports the suitability of the data reported in the Swedevox registry for use in evaluations and research.
Data quality is essential for both data analysis and the interpretation of clinical research. According to the principles of Good Clinical Practice applied in clinical drug trials, extensive data quality management with comparison between source data and reported data are performed to ensure correct results in the evaluation process of new therapeutic entities [18]. The findings of this study are in line with reported validations of other Swedish national registries [19–21].
The higher variability in LTMV data could reflect that the treatment is often initiated in relation to an acute decompensation of chronic respiratory failure, often requiring emergency and intensive care. Selection of blood gas values to report can be difficult as several assessments are often available from admission through acute management, decision to long-term treatment, to discharge. A similar problem may be relevant for the differences in weight, where patients starting LTMV may lose several kilograms during a short period of time due to mobilisation of oedema. We have not had clear instructions as to report weight before initiation or after a short period of LTMV treatment. However, there was no systematic differences in the reported values on the group-level which supports that also LTMV data are valid for health care quality assessment and research.
Reporting of the underlying diagnosis requiring LTMV showed the highest agreement for ALS, which probably reflects that most of these patients come with a precise diagnosis at their first contact with the LTMV clinic. As in OHS versus respiratory disease (consisting mainly of COPD), these diseases are known to frequently overlap and have an over-additive impact on respiration, and the distinction which of the two is the main contributor is in some cases difficult or even arbitrary. Furthermore, many of these patients start their therapy in an acute setting, before a firm diagnosis is established. This validation suggests that when using Swedevox data with OHS and respiratory disease, incorporating available data on BMI and lung function is likely to be valuable.
Strengths of the present analysis include that it evaluated clinically and scientifically relevant data for each treatment across multiple centres, and that the validation cohorts were representative for the remaining patients reported into the Swedevox registry. The validated variables and response categories were unchanged throughout the study period, and the registry has near-complete geographical coverage throughout Sweden. In addition, our validation study was blinded to the registered data in Swedevox. Finally, individuals reporting data into the registry differed at least in part from those who performed the validation process which increases the generalizability of our results.
Potential limitations of our study included the fact that data were only validated at baseline, as that is the main time point of interest for the registry, and rates of missing data are higher at follow-up. As the aim was to validate the registered data, we only included patients with complete data on the evaluated variables, and the rate of missing data in Swedevox was not assessed. However, characteristics were similar between the validation sample and all registered patients in Swedevox during the study period (supplemental tables s1–s3), supporting the generalizability of the findings to Sweden and similar settings.
Implications of the present findings include that Swedevox data closely mirrors the clinical patient characteristics and trajectories documented in the medical records, which supports the validity of using Swedevox data for clinical surveillance and follow-up of the therapies, nationwide comparisons of healthcare quality and for recommendations and clinical guidelines.
In conclusion, the present validation support that Swedevox data are valid, of high quality and suitable for use in research. Taken together with cross-linkage to national registry data, such as diagnosed diseases, medical procedures/surgery, hospitalisations and survival, with near-complete follow-up, as well as linkage with other national disease-specific quality registries, this poses strong research opportunities. The validity of Swedevox data are of fundamental importance for ongoing large-scale clinical trials [11, 16].
Supplementary material
Supplementary Material
Please note: supplementary material is not edited by the Editorial Office, and is uploaded as it has been supplied by the author.
Supplementary material 00340-2020.SUPPLEMENT
Acknowledgements
We extend our warm thanks to Kerstin Ström, Karlskrona, who initiated the registry in 1987, and to all nurses and staff who report to Swedevox and care for the patients.
Footnotes
This article has supplementary material available from openres.ersjournals.com
Data availability: Individual deidentified participant data (including data dictionaries) and the statistical analysis plan can be shared upon request to the corresponding author.
Author contributions: Concept and draft design: M. Ekström; final protocol and data collection: all authors; analyses and first manuscript draft: M. Ekström; revision and approval of final version to be published: all authors.
Conflict of interest: M. Ekström has nothing to disclose.
Conflict of interest: D. Albrecht has nothing to disclose.
Conflict of interest: S. Andersson has nothing to disclose.
Conflict of interest: L. Grote reports no conflict of interest for the actual study. Outside the submitted work, L. Grote reports a research collaboration with Bayer AG, nonfinancial support and other from Itamar Medical, Resmed, Philips and Astra Zeneca. In addition, Dr Grote has a patent on sleep apnoea therapy issued and licensed.
Conflict of interest: B. Kärrsten-Rundström has nothing to disclose.
Conflict of interest: A. Palm has nothing to disclose.
Conflict of interest: J. Theorell-Haglöw has nothing to disclose.
Conflict of interest: J. Wahlberg has nothing to disclose.
Conflict of interest: B. Midgren has nothing to disclose.
Support statement: The study was supported by the Swedish Association of Local Authorities and Regions; M. Ekström was supported by an unrestricted grant from the Swedish Research Council (Dnr: 2019-02081). Funding information for this article has been deposited with the Crossref Funder Registry.
- Received June 2, 2020.
- Accepted October 16, 2020.
- Copyright ©ERS 2021
This article is open access and distributed under the terms of the Creative Commons Attribution Non-Commercial Licence 4.0.
References
