Abstract
Randomised controlled trials (RCTs) are the gold standard for evaluating treatment efficacy in patients with obstructive lung disease. However, due to strict inclusion criteria and the conditions required for ascertaining statistical significance, the patients included typically represent as little as 5% of the general obstructive lung disease population. Thus, studies in broader patient populations are becoming increasingly important. These can be randomised effectiveness trials or observational studies providing data on real-world treatment effectiveness and safety data that complement efficacy RCTs.
In this review we describe the features associated with the diagnosis of asthma and chronic obstructive pulmonary disease (COPD) in the real-world clinical practice setting. We also discuss how RCTs and observational studies have reported opposing outcomes with several treatments and inhaler devices due to differences in study design and the variations in patients recruited by different study types. Whilst observational studies are not without weaknesses, we outline recently developed tools for defining markers of quality of observational studies. We also examine how observational studies are capable of providing valuable insights into disease mechanisms and management and how they are a vital component of research into obstructive lung disease.
As we move into an era of personalised medicine, recent observational studies, such as the NOVEL observational longiTudinal studY (NOVELTY), have the capacity to provide a greater understanding of the value of a personalised healthcare approach in patients in clinical practice by focussing on standardised outcome measures of patient-reported outcomes, physician assessments, airway physiology, and blood and airway biomarkers across both primary and specialist care.
Abstract
Observational studies can support RCTs in influencing clinical practice in the field of obstructive lung disease https://bit.ly/36YWu0W
Introduction
Intervention trials, such as randomised controlled trials (RCTs), and observational studies, such as registry studies have, until recently, been perceived as being distinct and mutually exclusive approaches to clinical research in respiratory medicine, as well as in other fields of medical research. Classical RCTs aim to establish the safety and efficacy of a treatment in the target patient population [1, 2], whereas classical epidemiology observational studies aim to ascertain how often diseases occur in different groups of people and why [3]. Additionally, epidemiological information is used to prepare and evaluate strategies to prevent illness and as a guide for the management of patients in whom disease has already developed [3]. Real-world observational studies with a prospectively recruited cohort aim to establish the effectiveness and safety of a treatment compared with others in a more general population of patients in a real-world, clinical practice setting, both with and without deliberate manipulation or intervention [1]. Furthermore, real-world studies enable exploratory research in broad patient populations that can be used to generate hypotheses, improve understanding of various aspects of disease and treatments, provide novel perspectives and challenge existing paradigms [1, 4].
The aim of this review is to evaluate the strengths and limitations of existing observational studies in assessing the effectiveness of pharmacological treatment in asthma and/or chronic obstructive pulmonary disease (COPD), in order to highlight key considerations for ongoing and future observational studies in obstructive lung disease.
Comparing RCTs with observational real-world studies
Observational studies and classical efficacy RCTs ask distinct research questions and thus employ different study methodologies and patient populations to answer them. Classical efficacy RCTs aim to compare the efficacy and safety of treatments within a patient population selected using strict inclusion criteria (e.g. exclusion of active smokers), with high disease severity (in terms of lung function impairment), good treatment adherence and good inhaler technique, thereby tightly controlling confounding factors. Although this level of internal validity and control makes it easier to identify the absolute benefit or lack of benefit of a treatment, it comes at the cost of external validity [5]; thus, results from efficacy RCTs may not be broadly generalisable to the wider population of patients with obstructive lung disease. Indeed, while RCTs remain the gold standard for evaluating treatments [2], the patients they include can represent as few as 5% of the general asthma/COPD population [6, 7].
In contrast, pragmatic RCTs aim to assess the differential benefit of a treatment in a broader patient population (e.g. patients with less severe lung function impairment and more comorbidities) in a normal ecology of care and with less intensive medical supervision compared with efficacy RCTs [8]. However, pragmatic RCTs still involve a higher organisation of clinical practice than that expected in a real-world setting.
With less intervention and organisation than efficacy RCTs or pragmatic RCTs, pure observational studies offer a more practical and cost-effective means to investigate the long-term outcome of a treatment in a broader patient population than that included in an RCT [5, 8].
As real-world studies differ substantially from efficacy RCTs in their objectives and approach, their study design often requires different considerations and many more patients are eligible for both pragmatic RCTs and observational studies compared with efficacy RCTs [1]. Such studies are seen as increasingly important for understanding treatment effectiveness in a broader patient population [8–11], thereby potentially informing future treatment management strategies.
Real-world studies can take many forms, including the following.
Classical epidemiological studies, e.g. trajectories of lung function in COPD [12], assessing the association between sleep-disordered breathing and asthma [13] management, morbidity and mortality of COPD in Sweden [14] and identifying COPD subtypes and corresponding biomarkers [15].
Retrospective studies using existing, routinely collected health data, such as electronic medical records or insurance claims, e.g. a study on predicting asthma attacks using real-world primary care data in the UK [16].
Post-marketing surveillance/phase IV studies monitor the real-world response to newly approved treatments, including real-world safety and mortality. These studies have helped identify and understand events such as the increased mortality rates observed amongst patients with asthma who received salbutamol (in Australia) and fenoterol (in New Zealand) during the 1980s [17–19].
Comparative effectiveness or safety studies assess the differential benefit of a treatment in broad patient categories to inform a clinical or policy decision by providing evidence for adoption of the intervention into a real-world setting [20, 21]; e.g. The Salford Lung Study (a pragmatic RCT) [9, 10, 22], the Novel START study [23] and the Lung Health Study [24].
In practical terms, real-world studies complement results from RCTs by providing a higher external validity once the efficacy and safety of a treatment has been confirmed under the strictly controlled conditions of an RCT [5, 8]. Tools, such the PRECIS-2, are available to describe the representativeness of a clinical trial compared with a real-world setting and are a valuable resource [25]. In addition, observational studies can be used to investigate aspects that RCTs cannot, such as prevalence and incidence of disease, aetiology, defining prognoses, disease impact, and burden and cost-effectiveness of treatment. Table 1 shows select examples of findings from real-world asthma and COPD data highlighting the different research questions that can be asked from pure observational studies and pragmatic trials covering treatment choice, the use of inhalers, biomarkers and clinical disease history.
Examples of findings from real-world asthma and COPD data highlighting the different research questions that can be asked from pure observational studies and pragmatic trials
Principal causes and factors associated with forced expiratory volume in 1 s decline, COPD and asthma, as established from observational studies
The principal causes and factors associated with forced expiratory volume in 1 s (FEV1) decline, COPD and asthma, as established from previous observational studies, are listed in table 2. Carefully conducted longitudinal studies have been instrumental in establishing causal relationships in obstructive lung disease, with case–control and cohort studies in the 1950s and 1960s firmly establishing cigarette smoking as the single greatest risk factor for lung cancer [48–50]. More recently, ECLIPSE (Evaluation of COPD longitudinally to identify predictive surrogate endpoints), a longitudinal study, was devised with the aim of describing the subtypes of COPD, defining predictive or surrogate markers of disease progression and, potentially, novel targets for therapeutic intervention [15].
The principal causes and factors associated with FEV1 decline, COPD and asthma, as established from observational studies
Despite having plateaued and even fallen in some regions, globally the prevalence of asthma has been increasing rapidly for several decades [51]. There is a strong genetic component in asthma, demonstrated by concordance of approximately 50% in monozygotic twins with asthma [52]; however, the speed of the increase in prevalence is thought to be too high to be accounted for by a genetic change alone and is therefore more likely to be related to environmental changes [53].
Comparing guidelines, RCTs and observational study outcomes in obstructive lung disease
Treatment options
Results from RCTs have indicated a benefit of adding low-dose oral theophylline to inhaled corticosteroid (ICS) therapy for COPD [94, 95]; however, UK National Institute for Health and Care Excellence (NICE) guidelines for COPD do not recommend theophylline as the first-choice of treatment [96], and the Global Initiative for Chronic Obstructive Pulmonary Disease (GOLD) report states that there is only limited and contradictory evidence for the use of low-dose theophylline [97]. In addition, results from the TWICS (theophylline with inhaled corticosteroids) pragmatic trial found no benefit of theophylline added to ICS over placebo in a real-world setting [46]. Similarly, a systematic review of RCTs for asthma demonstrated the superiority of ICS over leukotriene receptor antagonists (LTRA) for the management of asthma [98]; however, the ELEVATE (A pragmatic randomised single-blind controlled trial and economic evaluation of the use of leukotriene receptor antagonists in primary care at steps 2 and 3 of the national asthma guidelines) pragmatic study found no difference in effectiveness between ICS and LTRA [45]. These seemingly conflicting findings may be due, in part, to the different patient populations included and the lower adherence rate with ICS versus LTRA [45, 99].
In terms of treatment reduction, the Global Initiative for Asthma (GINA) recommends stepping down ICS/long-acting β2-agonist (LABA) dose once asthma control has been achieved for ≥3 months [100]. However, the FFLUX (A randomised pragmatic trial of changing to and stepping down fluticasone/formoterol in asthma) pragmatic trial that investigated the stepping-down of treatment in patients who were stable following 12 weeks of treatment, found that patients with a history of one or two exacerbations within 12 months prior to starting treatment were at increased risk of re-exacerbation [44]. This highlights the need for research beyond the outcomes of efficacy RCTs to be considered when guidelines are developed and in this specific case, the need for asthma exacerbation history to be considered in guiding clinicians in stepping-down of treatment.
Until relatively recently, the recommended treatment for asthma has been ICS maintenance treatment with as-needed short-acting β2-agonists (SABAs) [101]. However, real-world data have found that patients typically underuse ICS and overuse SABA [102]. This has led to the observation that overuse of SABA is associated with an increase in all-cause mortality risk in patients with asthma [103]; the subsequent revision of the guidelines to recommend combined ICS/SABA as needed demonstrates how the outcomes of observational studies are influencing global guidelines [100].
Real-world data may also provide complementary evidence to support findings from RCTs. The Salford Lung Study pragmatic trial successfully demonstrated the real-world effectiveness of fluticasone furoate/vilanterol treatment for maintenance therapy of COPD [10] and asthma [9], adding to the findings from previous RCTs.
Some studies have also suggested limited value or even harm of certain therapies in COPD. An observational nested case–control study of patients with COPD being treated with LABA and ICS from registry data over 4.5 years found that the addition of the long-acting muscarinic antagonist (LAMA) tiotropium was associated with an increased cardiovascular risk in patients with COPD [104]. However, none of the recent fixed triple combination registration trials have seen this effect and in the three-year ASCENT (Evaluate the effect of aclidinium bromide on long-term cardiovascular safety and exacerbations in moderate to very severe COPD patients) RCT of patients with COPD and high cardiovascular risk, there was no increase in the risk of cardiovascular events for patients receiving the LAMA aclidinium compared with placebo [105].
Inhaler device, technique and adherence
RCTs typically ensure that patients demonstrate correct inhaler technique and adhere to their treatment; thus, results from RCTs reflect the efficacy of inhalers under a near-perfect technique and adherence rate [106, 107]. However, inhalation errors in a real-world setting have been shown to increase the risk of poor treatment outcomes, such as hospitalisation, medication use and symptom control [35, 36, 107]. In addition, mixing inhaler devices may lead to worse COPD outcomes than when single devices or devices requiring the same inhalation technique are used [29, 30]. Thus, results from real-world studies emphasise the importance of ensuring proper inhaler technique to maximise treatment success in both asthma and COPD. Other examples of findings from real-world evidence in asthma/COPD can be found in table 1.
With regard to specific inhaler types, according to the recommendations of the British Thoracic Society Scottish Intercollegiate Guidelines Network [108] and results from interventional RCTs, dry powder inhalers (DPI) are as effective as pressurised metered-dose inhalers (pMDI) for the delivery of ICS treatment [109, 110]. This is supported by a recent study utilising the Korean Health Insurance Claims database, which found a comparable clinical and cost efficiency between patients with asthma who switched from a DPI to a pMDI versus patients who remained on a DPI [28]. However, other results have been more conflicting [35, 36, 106, 107, 111] and real-world studies from the UK have suggested that pMDIs are superior to DPIs in both asthma [31] and COPD [32], illustrating that the outcomes of observational studies can still be conflicting and the importance of understanding the different methodologies and analyses used.
Weaknesses of observational studies
It is important to note that despite the many advantages of observational studies, as with all study designs, the methodologies employed are subject to specific biases, including selection bias (systematic differences between baseline characteristics of the groups that are compared) and detection bias (systematic differences between groups in how outcomes are determined) [112]. Studies utilising electronic health records are further susceptible to a degree of inaccuracy and incompleteness; such records are typically collected for routine medical purposes and can lack the quality, detail and accuracy typically required for research purposes [113].
In enrolling a broad patient population, the analysis of data generated from observational studies is complicated by confounding factors such as confounding by indication; i.e. most patients receiving medication in an observational study have been formerly diagnosed by a doctor whereas those without the medication have not, despite otherwise appearing almost identical [114]. Another factor which must be considered is the avoidance of immortal time balance, which can be the consequence of incorrect handling of the period between study entry and treatment initiation in time-to-event analyses [115]. For time-dependent confounders, such as body mass index, which is a risk factor for asthma that may lead to reduced physical activity and is also affected by prior levels of physical activity, the parametric g-formula can be used in place of conventional regression approaches [116, 117].
Biases in observational studies can be significantly reduced by using a prospective study design and a predefined statistical analysis plan [5, 118]. In addition, tools such as the Risk of Bias in Non-randomised Studies of Interventions (ROBINS-I) [119] and Newcastle-Ottawa Scale (NOS) for assessing the quality of non-randomised studies [120] may also be beneficial in minimising bias. However, while statistical adjustment and matching can be used to minimise confounding effects [5, 118], factors which are not accounted for, and thus not recorded within the study, are likely to remain. It should be noted that RCTs are often also affected by bias, such as selection and information bias, although this is not always recognised.
Markers of quality for observational studies
Despite their shortcomings, clinical guidelines still place a greater emphasis on results from classical RCTs than from observational studies [8]. Indeed, traditional tools for rating quality of studies such as the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) also downgrade observational study designs [8].
To achieve greater integration of real-world evidence into the development programmes of new drugs, it is vital that observational studies are subjected to standards that are as equally rigorous as those devised for classical RCTs [8]. There is, therefore, a need to standardise the quality of real-world evidence. Recently, a joint task force between the Respiratory Effectiveness Group and the European Academy of Allergy and Clinical Immunology (EAACI) developed a standardised tool for quality appraisal of comparative effectiveness studies, the REal Life EVidence AssessmeNt Tool (RELEVANT; www.regresearchnetwork.org/relevant-tool-2) [121]. The tool incorporates 21 quality checklist items, of which 11 primary items determine a study's suitability for guideline development and 10 secondary items are for general appraisal of the study. Quality appraisal using the RELEVANT tool on selected examples of comparative effectiveness studies are presented in table 3; similar tools are already available for evaluating observational studies [122, 123].
Author's appraisal of selected comparative effectiveness studies using RELEVANT 2.0 tool [121]
Why we need both RCTs and observational studies
Comparing RCT and observational study data by adjusting and aligning patient data has further highlighted the importance of using both study types to assess the effect of a treatment. A number of studies on the use of statins in patients with COPD have indicated that statins may provide additional benefits in terms of improving lung function and reducing risk of exacerbation, hospitalisation and death [124–126], potentially through reduction of inflammation [127]. Of particular interest, the STATCOPE (Simvastatin for the prevention of exacerbations in moderate-to-severe COPD) RCT found that statins had no impact on exacerbation risk, lung function, or on general or disease-specific quality of life in patients with COPD [128]. In contrast, an observational study by Ingebrigtsen et al. [129] found that statins did reduce exacerbation risk. However, when these same observational data were adjusted to align the patients with those from the STATCOPE RCT, statins were found to provide no additional benefit in patients with COPD. Due to the inherent differences in the patient populations of RCTs and observational studies, as previously described, this finding clearly demonstrates why both RCTs and observational studies are needed to form a complete picture of treatment effect.
Future prospects in real-world evidence in asthma/COPD
Several complex observational studies in asthma/COPD have contributed to a greater understanding of the heterogeneity of the asthma/COPD population in a real-world setting, including COPDGene [130], ECLIPSE [15], SPIROMICS (Subpopulations and intermediary outcomes in COPD study; U-BIOPRED: Unbiased biomarkers in prediction of respiratory disease outcomes) [131] and U-BIOPRED [132]. These studies have led to an increasing recognition of the importance of personalised healthcare and the value of endotype-driven assessment and management [133]. However, to date, both RCTs and real-world studies have largely examined the effects of pharmacological treatment at a population level. Thus, although treatment has been shown to have a statistically significant impact on symptoms, exacerbations and airflow obstruction, the scale of the effects at the group level are often limited, suggesting that not all patients may gain the same effect from treatment. Thus, as we enter an era of personalised medicine, there is a need to identify the individual patient factors that are associated with treatment response.
The recent shift towards a treatment approach guided by treatable disease characteristics, or traits [134], that is less dependent on conventional diagnostic labels, has highlighted a lack of studies that span both COPD and asthma across a broad range of severities. In order to provide a greater understanding of the value of a personalised healthcare approach in patients in clinical practice, there is a need for large-scale, inclusive observational studies with standardised outcome measures and a focus on patient-reported outcomes, physician assessments, airway physiology and blood and airway biomarkers across both primary and specialist care. The NOVEL observational longiTudinal studY (NOVELTY) study (NCT02760329) is one such study that aims to address this need [135]. NOVELTY is a global (19 countries), 3-year prospective, observational study of >12 000 patients with a diagnosis or suspected diagnosis of asthma and/or COPD that aims to describe patient characteristics, treatment patterns and burden of illness, and to identify the clinical phenotypes and molecular endotypes (based on biomarkers and/or clinical parameters) that are associated with differential outcomes for symptom burden, clinical evolution and healthcare utilisation over time. It is expected that the majority of patients enrolled in NOVELTY would not have been eligible for inclusion in most RCTs, therefore NOVELTY offers the prospect of investigating disease mechanisms and outcomes in a more clinically relevant population than that provided by a classical RCT.
Conclusions
Real-world evidence is capable of providing valuable insights into disease mechanisms and management; however, due to the potential for producing large amounts of data and analyses compared with RCTs, it is vital that they are designed with clear research questions in mind. These research questions may demand different methodologies and, as such, will guide the type of study that is required. This will help to challenge perceptions that real-world evidence is solely for the evaluation of safety/epidemiology, and will demonstrate that they can also inform on patient outcomes if designed with clear research questions. Furthermore, due to the inclusion of a broader range of patients than RCTs, real-world studies require a much greater understanding of confounders and modifiers of effects compared with RCTs to aid interpretation of their findings.
Observational real-world studies are a vital component of research into obstructive lung disease, and well-designed observational studies can support pivotal RCTs and provide evidence that has the potential to influence clinical practice. Although observational studies are subject to specific challenges, with the aid of recently developed quality standard tools, these challenges can be factored into study design to produce high-quality results. In future, well-designed, real-world studies that include a broad range of patients (in terms of geographical location, care setting and severity level) across both asthma and COPD diagnoses will be instrumental in supporting a more personalised, endotype-driven approach to the assessment and management of patients with obstructive lung disease.
Acknowledgements
Editorial support under the direction of the authors was provided by Richard Knight, CMC Connect, McCann Health Medical Communications, and funded by AstraZeneca in accordance with Good Publication Practice guidelines. The first draft of the manuscript was written in three sections by J. Vestbo, C. Janson and D. Price. Editorial support specifically for D. Price was provided by Antony Hardjojo of the Observational and Pragmatic Research Institute, Singapore. J. Vestbo is supported by the NIHR Manchester BRC.
Footnotes
Conflict of interest: J. Vestbo reports personal fees from AstraZeneca for co-chairing the NOVELTY study, from which the idea for this article arose, during the conduct of the study; honoraria for presenting and/or advising on COPD from AstraZeneca, an unconditional biomarker grant and honoraria for presenting and/or advising on COPD from Boehringer Ingelheim, honoraria for presenting and/or advising on COPD from Chiesi, an honorarium for advising on COPD from GSK, and honoraria for presenting and/or advising on COPD fees from Novartis, outside the submitted work; and his son is an employee of Chiesi (Denmark).
Conflict of interest: C. Janson has nothing to disclose.
Conflict of interest: J. Nuevo is an employee of AstraZeneca.
Conflict of interest: D. Price reports grants from AKL Research and Development Ltd, personal fees from Amgen, grants and personal fees from AstraZeneca, grants and personal fees from Boehringer Ingelheim, grants from British Lung Foundation, grants and personal fees from Chiesi, grants and personal fees from Circassia, personal fees from Cipla, personal fees from GlaxoSmithKline, personal fees from Kyorin, personal fees from Merck, grants and personal fees from Mylan, grants and personal fees from Mundipharma, grants and personal fees from Napp, grants and personal fees from Novartis, grants and personal fees from Pfizer, grants and personal fees from Regeneron Pharmaceuticals, grants from Respiratory Effectiveness Group, grants and personal fees from Sanofi Genzyme, grants and personal fees from Teva, grants and personal fees from Theravance, grants from UK National Health Service, grants and personal fees from Zentiva (Sanofi Generics), non-financial support from Efficacy and Mechanism Evaluation programme, non-financial support from Health Technology Assessment, outside the submitted work; stock/stock options from AKL Research and Development Ltd, which produces phytopharmaceuticals; and owns 74% of the social enterprise Optimum Patient Care Ltd (Australia and UK) and 74% of Observational and Pragmatic Research Institute Pte Ltd (Singapore).
Support statement: The development of this manuscript was funded by AstraZeneca. Funding information for this article has been deposited with the Crossref Funder Registry.
- Received January 28, 2020.
- Accepted June 1, 2020.
- Copyright ©ERS 2020
This article is open access and distributed under the terms of the Creative Commons Attribution Non-Commercial Licence 4.0.
References
