Skip to main content

Main menu

  • Home
  • Current issue
  • Early View
  • Archive
  • Authors/reviewers
    • Instructions for authors
    • Submit a manuscript
    • COVID-19 submission information
    • Institutional open access agreements
    • Peer reviewer login
  • Alerts
  • Subscriptions
  • ERS Publications
    • European Respiratory Journal
    • ERJ Open Research
    • European Respiratory Review
    • Breathe
    • ERS Books
    • ERS publications home

User menu

  • Log in
  • Subscribe
  • Contact Us
  • My Cart

Search

  • Advanced search
  • ERS Publications
    • European Respiratory Journal
    • ERJ Open Research
    • European Respiratory Review
    • Breathe
    • ERS Books
    • ERS publications home

Login

European Respiratory Society

Advanced Search

  • Home
  • Current issue
  • Early View
  • Archive
  • Authors/reviewers
    • Instructions for authors
    • Submit a manuscript
    • COVID-19 submission information
    • Institutional open access agreements
    • Peer reviewer login
  • Alerts
  • Subscriptions

Safety data in randomised real-world evidence studies: Salford Lung Study learnings

Catherine Harvey, Ashley Woodcock, Jørgen Vestbo, Courtney Crim, Lucy Frith, Nawar Diar Bakerly, John P. New, Claire Williams, Hanaa Elkhenini, Nasir Majeed, Glenn Cardwell, Susan Collier, Loretta Jacques, Joanne Fletcher
ERJ Open Research 2021 7: 00966-2020; DOI: 10.1183/23120541.00966-2020
Catherine Harvey
1GlaxoSmithKline plc, Uxbridge, Middlesex, UK
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • For correspondence: Cath_M_Harvey@gsk.com
Ashley Woodcock
2Manchester Academic Health Sciences Centre, The University of Manchester, Manchester, UK
3Manchester University NHS Foundation Trust, Manchester, UK
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jørgen Vestbo
2Manchester Academic Health Sciences Centre, The University of Manchester, Manchester, UK
3Manchester University NHS Foundation Trust, Manchester, UK
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Jørgen Vestbo
Courtney Crim
4GlaxoSmithKline plc, Research Triangle Park, NC, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Lucy Frith
1GlaxoSmithKline plc, Uxbridge, Middlesex, UK
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Nawar Diar Bakerly
2Manchester Academic Health Sciences Centre, The University of Manchester, Manchester, UK
5Salford Royal NHS Foundation Trust, Salford, UK
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
John P. New
2Manchester Academic Health Sciences Centre, The University of Manchester, Manchester, UK
5Salford Royal NHS Foundation Trust, Salford, UK
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Claire Williams
6NorthWest EHealth, Manchester, UK
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Hanaa Elkhenini
7Pennine Acute Hospitals NHS Trust, Oldham, UK
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Nasir Majeed
5Salford Royal NHS Foundation Trust, Salford, UK
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Glenn Cardwell
1GlaxoSmithKline plc, Uxbridge, Middlesex, UK
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Susan Collier
1GlaxoSmithKline plc, Uxbridge, Middlesex, UK
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Loretta Jacques
1GlaxoSmithKline plc, Uxbridge, Middlesex, UK
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Joanne Fletcher
1GlaxoSmithKline plc, Uxbridge, Middlesex, UK
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Article
  • Figures & Data
  • Info & Metrics
  • PDF
Loading

Abstract

Evidence to support clinical decision making must be based on safety data that have been captured, analysed and interpreted in a robust and reliable way. Randomised real-world evidence (RRWE) studies provide the opportunity to evaluate the use of medicines in patients and settings representative of routine clinical practice. However, elements that underpin the design of RRWE studies can have a significant impact upon the analysis, interpretation and implications of safety data.

In this narrative review, we use data from the Salford Lung Study; two prospective, 12-month, open-label, parallel-group, phase III randomised controlled trials conducted in primary care in the UK; to highlight the importance of capturing treatment modifications when attempting to evaluate safety events according to actual treatment exposure.

We demonstrate that analysing safety data by actual treatment received (i.e. accounting for the treatment modifications that occur routinely in the primary care setting) provides additional insight beyond analysing according to randomised treatment strategy only.

It is therefore proposed that understanding of safety data from RRWE trials can be optimised by analysing both by randomised group and by actual treatment received.

Abstract

Using results from the Salford Lung Study, this review demonstrates that the understanding of safety data from randomised real-world evidence studies can be optimised by analysing both actual treatment received and randomisation group https://bit.ly/3p7zPXU

Introduction

Treatment guidelines and management algorithms for a variety of conditions, including the respiratory diseases chronic obstructive pulmonary disease (COPD) and asthma, are largely based on data from efficacy randomised controlled trials (RCTs) [1, 2]. Traditional double-blind RCTs typically recruit patients according to strict inclusion/exclusion criteria and are conducted under highly controlled conditions, so as to minimise bias due to confounding variables. However, such trials do not properly reflect how medicines are used in everyday clinical practice and under-represent characteristics within the wider disease population. Consequently, generalising findings from traditional double-blind RCTs to patients in routine clinical practice can be problematic [3, 4]. For example, a previous study reported that only 5.4% of asthma patients satisfied four commonly used criteria for entry to an asthma efficacy RCT (absence of co-morbidity, forced expiratory volume 50–85% of predicted, present or historical reversibility of 12% in the last year, non-smoker or, if ex-smoker, a smoke burden less than 10 pack–years); the proportion of eligible patients further reduced to 3.3% when eligibility was made more stringent by the inclusion of two additional common criteria [3].

There is a need for evidence derived from studies conducted in conditions closer to routine clinical practice [4, 5]. RCTs conducted in primary care with elements of real-world design, referred to hereafter as randomised real-world evidence (RRWE) studies, provide much needed insights into the effectiveness and safety of medicines in patients and clinical settings that represent routine clinical practice. However, the apparent simplicity in the design of RRWE studies, paradoxically, adds complexity to the understanding of the safety data collected (outlined in table 1). In this narrative review article, we discuss the analysis, interpretation and implications of safety data obtained in RRWE studies, using the phase III Salford Lung Study (SLS) as an illustrative example [6–10].

View this table:
  • View inline
  • View popup
TABLE 1

Study design features affecting the interpretation of safety data in randomised real-world evidence studies versus traditional efficacy randomised controlled trials

Safety data in the SLS

The SLS programme comprised two prospective, 12-month, open-label, parallel-group, phase III effectiveness RCTs conducted in primary care in and around Salford, UK. The SLS enrolled a large population of patients with a general practitioner's diagnosis of either COPD (SLS COPD) or asthma (SLS asthma). The trials evaluated the effectiveness and safety of initiating once-daily inhaled fluticasone furoate/vilanterol (FF/VI) compared with continuation of usual care (figure 1) [9, 10]. Randomisation to initiate FF/VI or continue usual care was stratified in both trials; specifically, patients were stratified by baseline maintenance therapy, to minimise confounding that might result from differences between the treatment groups.

FIGURE 1
  • Download figure
  • Open in new tab
  • Download powerpoint
FIGURE 1

Overview of design of Salford Lung Study (SLS) COPD and SLS asthma groups. #: randomisation stratified by presence/absence of a COPD exacerbation in the previous 12 months and baseline intended COPD maintenance therapy. +: randomisation stratified by baseline Asthma Control Test (ACT) score (≥20, 16–19, ≤15) and baseline intended asthma maintenance therapy. AQLQ: Asthma Quality of Life Questionnaire; BD: bronchodilator; CAT: COPD Assessment Test; EHR: electronic health record; FF/VI: fluticasone furoate/vilanterol; GP: general practitioner; ICS: inhaled corticosteroid; LABA: long-acting β2-agonist; LAMA: long-acting muscarinic antagonist; MARS-A: Medical Adherence Report Scale for Asthma; UC: usual care; WPAI: asthma: Work Productivity and Activity Impairment questionnaire.

In the SLS, more than 7000 patients provided written consent for their data to be used for safety monitoring [11]. Safety data (serious adverse events (SAEs), both treatment-related and non-treatment-related, and non-serious adverse drug reactions (ADRs; non-serious adverse events considered to be treatment-related in the opinion of the investigator)) were collected by near real-time review of patients’ electronic health records (EHRs) following electronic trigger alerts and monitoring of primary care data. Safety was assessed daily during the 4.5 years of the study. A detailed description of the safety monitoring and unique methods of data collection in the SLS has been published previously [12]. Crucially, as safety data could be monitored remotely using the EHR, study-specific visits were not mandated and patients could continue with their lives as usual, unlike in traditional efficacy RCTs where potentially inconvenient research site visits are required.

Safety endpoints included SAE frequency and type and there was specific examination of SAEs of special interest (SAESI), defined as known class effects of inhaled corticosteroids (ICS) and/or long-acting β2-agonists (LABA). SAESI groupings were defined a priori using groups of related Medical Dictionary for Regulatory Activities (MedDRA) event terms (by MedDRA structure, where possible), ensuring all relevant events were captured. One example was pneumonia, a known and accepted class effect of ICS use in patients with COPD. The pneumonia SAESI group was defined a priori by 69 MedDRA preferred terms; any verbatim term for an SAE that mapped to one of these terms was therefore counted as a pneumonia SAESI. This ensured capture of all pneumonia events (e.g. those where an infective agent was part of the reported pneumonia term).

Understanding the impact of RRWE study design on safety data

Patient population

The heterogeneity of the patient population enrolled in RRWE studies has implications for understanding safety data. Including patients with multiple comorbidities and with many comorbid disease-related events can lead to far more SAEs than are usually seen in traditional RCTs, which typically exclude such patients. For example, the SLS did not exclude patients with a comorbid cancer diagnosis unless they were predicted to live for less than a year following randomisation. This resulted in more cancer-associated SAEs than would be observed in a traditional RCT. Although bias is possible in an open-label trial, randomisation reduces the risk for bias, as well as ensuring that reporting of SAEs that are non-treatment related should be balanced across groups, and indeed in the SLS there were no observed differences between treatment groups. Therefore, when SAEs are analysed, patient characteristics such as comorbidities should be considered.

Treatment modifications: analysing by actual versus randomised treatment

Modification of treatment is common in everyday clinical practice. This can be “stepping up” treatment to provide better disease control and management (e.g. addition of a new class of drug), or “stepping down” treatment in situations where disease control has been maintained for a specific period of time (e.g. removal of one class of drug) [13]. Modification could also involve switching to a different treatment altogether. Physicians and patients regularly modify treatment for asthma and COPD for various reasons, including symptom control, patient preference, familiarity, cost, reimbursement status, adverse reactions, convenience, device characteristics, adequate inhaler technique, disease progression and potential drug interactions. As such, there is a need to both allow and capture these treatment modifications in studies intended to more closely reflect usual clinical practice. Accordingly, treatment modifications are an integral element in the design, execution and analytical approach of RRWE studies. By capturing treatment modifications, RRWE studies can provide evidence founded on actual treatment received (true exposure risk) [10], an approach that could be increasingly adopted in future effectiveness studies. Traditional RCTs generally do not allow for treatment modifications: participants must adhere to their randomised treatment throughout the study, and this is strictly monitored. Therefore, understanding the design of RRWE studies and whether participants have the option to modify their treatment directly impacts upon our ability to understand and interpret safety data.

Patients were able to modify their treatment in the SLS. This had far-reaching implications for the analysis and interpretation of the collected safety data. Patients who were randomised to the FF/VI group initiated treatment with FF/VI but were allowed to continue FF/VI or to change to any other suitable medicine for their condition while remaining in the study and being retained for analysis purposes in their original FF/VI randomisation group. Although randomised to initiate FF/VI, patients could have been taking a different medicine at any subsequent time (ICS, LABA or long-acting muscarinic antagonist (LAMA); either alone or in different combinations, according to the study protocol). If patients who modified their treatment continued with the new treatment for several months, it is questionable whether any later adverse events can be considered part of the FF/VI “treatment period” rather than the “actual” treatment being taken at the time of the event. Here, the FF/VI randomisation group is not equivalent to an FF/VI treatment group; comparison of safety data between the randomisation groups in a RRWE study does not equate to a treatment comparison, but to a strategy comparison.

Analysis of safety data by actual treatment received is itself not without potential bias. Firstly, there could be a carry-over effect of risk; in the SLS for example, the carry-over of pneumonia risk, which was addressed in the actual treatment analysis by applying a risk window. Secondly, bias could be introduced as a result of the implications associated with asymmetric treatment modification. In the SLS, patients not content with FF/VI treatment will have switched to an alternative treatment, resulting in a comparison of safety events between patients who remained on FF/VI throughout the study and a mixed group of patients. This latter group included patients who changed from FF/VI to an alternative treatment during the study, patients who continued the treatment they were receiving before the study and also patients in the usual care group who modified their prior study treatment in some way. The asymmetry of treatment modification in the SLS could favour treatment stability in the FF/VI group, but there is also an inherent stability for patients who remained on their prior (usual care) medication during the study. Similarly, when analysing safety data by randomised treatment group, there is an inherent stability built into the usual care comparator group versus the FF/VI randomisation group; the bias here will be for the stability in the usual care group. Importantly, additional bias to the safety analysis could be introduced by treatment change temporally related to adverse events or SAEs.

Use of a comparator group in RRWE studies: what constitutes usual care?

To ensure relevance to everyday clinical practice, the comparator for an investigational medicine in a RRWE study is typically a usual care group. This group can receive multiple different drugs, administered alone or in a variety of combinations. In the SLS, the random allocation was patients being randomised either to initiate FF/VI or to continue their previous treatment (usual care). For asthma and COPD, usual care can mean a variety of treatments differing by class (e.g. ICS, LABA, LAMA alone or in combination, according to the study protocol), dose or brand. In the SLS, patients randomised to the usual care group were allowed, at any time and for any reason, to modify their treatment to any other suitable usual care medication (but not to FF/VI), whereas patients randomised to initiate FF/VI could modify their treatment to usual care. The potential for continuous variation in the treatments being taken by patients randomised to the FF/VI and usual care groups complicates data analysis: it precludes direct comparison of randomisation groups from being equated with the safety of one specific treatment relative to another.

Open-label design

When an open-label study seeks to compare a new medicine with existing therapies, patients and physicians will be aware that they are receiving something new, and this brings the potential for reporting bias [14]; it might be more likely that new adverse events will be assigned causally to a new medicine than to an existing older one. Patients and physicians become familiar with asthma and COPD medications used for the long-term management of these conditions, and consequently, patients could establish preference for a particular medication or inhaler type. These potential issues associated with investigator evaluation and attribution of the causality for a given adverse event apply to both RRWE studies and traditional efficacy RCTs. In this respect, future studies may benefit from a blinded adverse event reporting team for causality and severity.

The issues surrounding reporting bias in open-label studies is potentially exemplified in the SLS, which commenced prior to licensing of FF/VI in the UK. Although a number of blinded studies have shown an adverse event profile for FF/VI similar to other ICS/LABA therapies, the SLS assigned causality of adverse events to FF/VI use (i.e. more on-treatment ADRs/treatment-related adverse events with FF/VI than for usual care) (online supplementary table S1) [15, 16]. Whilst this could be a finding of the RRWE that was not reported in RCTs, it could also be a result of reporting bias. Further RRWEs could help to provide additional data to clarify this disparity. While the comparative number of SAEs in the different treatment groups should be considered, the incidence of events that are likely to be related to these medicines (based on the likely class effects of ICS or LABA, those selected as SAESI in the SLS) was comparable between randomisation groups (online supplementary table S2) [15, 16].

Safety data in RRWE studies: examples from the SLS

Collecting information relating to treatment modifications was an important feature of the SLS study design. From the perspective of the randomisation groups, treatment modifications in the SLS were asymmetric. Patients randomised to FF/VI could change treatment to any other suitable treatment for any reason (i.e. commence treatment considered to be usual care), while patients randomised to usual care were free to modify their treatment to any other usual care treatment (except FF/VI). Any patient who modified their treatment remained in their original randomisation group for analysis purposes, including those who changed from treatment with FF/VI to another treatment.

In SLS COPD [9], 24% of patients randomised to initiate FF/VI underwent at least one treatment modification (figure 2); the majority changed back to their previous medicines but nevertheless remained in the FF/VI group. In the usual care randomisation group, 11% of patients modified their treatment, but this meant that they remained on a usual care treatment while remaining in the usual care randomisation group. In SLS asthma [10], similar rates of treatment modifications were observed: 22% of patients in the FF/VI randomisation group and 18% in the usual care randomisation group modified their treatment regimen (figure 2); again, most of the patients in the FF/VI group changed back to the medicine they were taking at study entry. These findings result from the open-label design of the SLS and possibly reflect the apparent preference of some patients and/or physicians for familiar treatments. Our experiences from the SLS illustrate the importance of capturing modifications of treatment when attempting to evaluate safety according to actual treatment exposure. Comparing randomisation groups is not the same as comparing the safety of FF/VI versus the safety of usual care. It is comparing the strategy of initiating treatment with FF/VI versus the strategy of continuing usual care.

FIGURE 2
  • Download figure
  • Open in new tab
  • Download powerpoint
FIGURE 2

Treatment modifications by randomised treatment group during the 12-month study period in Salford Lung Study (SLS) COPD and SLS asthma groups. A key design characteristic of the SLS was the ability for patients’ treatment to be modified during the 12-month study period. Treatment modifications included a change in class of medication, an increase/decrease in medication dose or a change in brand of medication. Patients who initiated treatment with fluticasone furoate/vilanterol (FF/VI) were permitted to modify treatment to usual care (UC); however, patients who initiated treatment with UC were not permitted to initiate treatment with FF/VI. #: total study population (intent-to-treat).

In the SLS, we analysed safety data according to both randomisation group and actual treatment received. The former approach evaluated the effectiveness and safety of the strategy of initiating FF/VI (randomisation to the FF/VI group) versus continuing previous usual care (randomisation to the usual care group). The latter approach determined the safety of treatment being taken at the time of onset of the event, regardless of randomisation group. Here, there is a juxtaposition of exposure risk and the occurrence of an adverse event; if a patient is not currently exposed to the treatment and has an event, it cannot reasonably be assigned to having occurred while “on treatment” with that product. However, this is also inadequate as some adverse events might be associated with cumulative recent exposure, rather than immediate exposure - pneumonia for example. In the SLS, an “at-risk” window was pre-planned for the evaluation of pneumonia events; the relevant risk window of treatment at the time of the event for pneumonia was the treatment taken for the majority of the 28-day period prior to onset of the event. Such rules for assigning safety events to a risk period of actual treatment need to be determined a priori in RRWE studies, with differences in time of treatment initiation/randomisation and adverse event onset, and possible drug exposure risk levels being considered in each case.

At the request of regulatory authorities, the SAESI of pneumonia was statistically compared to determine non-inferiority by randomisation group rather than by actual treatment in the SLS. Although characterising the incidence of SAESI of pneumonia had not been an original objective of the SLS, the real-world design and use from the outset of a comprehensive EHR system to monitor safety (which allowed access to all collected healthcare information from the point of patient consent) enabled pneumonia events to be collected both retrospectively and prospectively while the study was ongoing. In SLS COPD, there was a small numerical difference in the number of patients experiencing at least one pneumonia SAESI: 94 (7%) patients in the FF/VI randomisation group versus 83 (6%) in the usual care randomisation group (table 2); statistically, being randomised to initiate FF/VI was not different to being randomised to continue usual care, with an incidence ratio of 1.1 (0.9–1.5) [9, 15]. The total number of pneumonia SAESIs among both groups was 211; patients randomised to the FF/VI group and to the usual care group experienced 104 (52%) and 97 (48%) events, respectively [15].

View this table:
  • View inline
  • View popup
TABLE 2

Pneumonia and cardiovascular effects: serious adverse events of special interest (SAESI) analysed by randomised treatment group and by actual treatment in Salford Lung Study (SLS) COPD and SLS asthma

Pneumonia events can also be expressed as a rate per 1000 patient-years. In the SLS, this means a comparison of the rate of events per 1000 years of the strategy of randomising to initiate FF/VI versus the strategy of continuing usual care. For events related to exposure, this is a useful comparison of rates. As discussed earlier, comparing randomisation groups in the SLS is not the same as comparing one treatment with another. In separate analyses of SLS COPD not previously published, the incidence of pneumonia SAESI was evaluated allowing for permitted treatment modifications that occurred during the study, i.e. analysis by actual treatment (for pneumonia, during the risk window). Patients in the FF/VI randomisation group, at different times, could have been receiving FF/VI, or have switched to another treatment. A more appropriate comparison is the rate of events per 1000 patient-years of the actual treatment received at the time of the event. When actual treatments are compared in this way, the rates of pneumonia per 1000 patient-years were as follows: FF/VI 61.2 versus other ICS/LABA 64.0, and FF/VI+LAMA 60.9 versus other ICS/LABA/LAMA 92.7 (table 2) [15].

In SLS asthma, there was also a small observed difference in the number of patients with pneumonia SAESI when comparing randomisation groups: 23 (1%; 24 events) for FF/VI and 16 (<1%; 18 events) (table 2) [10, 16]. However, analysis by actual treatment, taking into consideration the treatment modifications during the study, revealed the same number of pneumonia events in both treatments (21 each) [10]. When pneumonia events were analysed by actual treatment risk, the event rate per 1000 patient-years was: FF/VI 10.7 versus ICS 5.9 and ICS/LABA 9.7 (table 2) [10, 16].

The one-way treatment modification in the SLS that permitted modifying treatment from FF/VI to other treatments, but not allowing those randomised to usual care to receive FF/VI, distorted the pneumonia rates and represented a potential bias against FF/VI when pneumonia SAESI data were analysed according to randomisation group. This was shown to be the case in SLS COPD when pneumonia was analysed by actual treatment received.

A similar phenomenon can be seen when examining other SAESI in the SLS, e.g. cardiovascular events, which were evaluated using standardised MedDRA outputs. In SLS COPD, when analysed by randomised treatment, 108 (8%) patients in the FF/VI randomisation group experienced a cardiovascular event versus 107 (8%) patients in the usual care randomisation group (table 2) [9]. However, when analysed by actual treatment, rates per 1000 patient-years for the frequency of cardiovascular events were: FF/VI 110.1 versus other ICS/LABA 167.6, and FF/VI+LAMA 92.8 versus other ICS/LABA/LAMA 137.5 [15]. Whilst prior history of cardiovascular disease was not a stratification criterion at randomisation, it is suggestive that patients actually taking FF/VI were at lower risk than those taking other ICS/LABAs. Similarly, in SLS asthma, when analysed by randomisation group, 46 (2%) patients in both the FF/VI and usual care randomisation groups had a cardiovascular event, but rates per 1000 patient-years of actual treatment were: FF/VI 23.3 versus ICS 31.6, and ICS/LABA 28.6 (table 2) [16]. This was also observed for SAESIs of lower respiratory tract infections excluding pneumonia SAEs and incidence of decreased bone mineral density and associated fractures SAESIs.

This is an important observation. When treatment modifications are allowed in RRWE studies, analysis by randomisation group does not provide a comparison of the safety of one treatment versus another, and so analysis must be performed by actual treatment at the time of an event in order to understand the relative risk of the treatments. Although the comparison of pneumonia and cardiovascular events according to actual treatment resulted in fewer events with FF/VI compared with other classes of ICS/LABA (with the exception of pneumonia in SLS asthma, where the converse was true), this finding would have been overlooked if the study safety data had been analysed by randomisation group only.

Implications of safety data derived from RRWE studies

As noted earlier, data from the SLS highlights the potential for misinterpretation of safety data derived from RRWE studies. This is most apparent when analyses are conducted solely by randomised treatment group, and do not necessarily reflect true drug exposure during a study in which, by design, treatment modifications occur routinely. However, caveats also exist for analyses conducted according to actual treatment received, such as ignoring potential carry-over effects for risk and the potential biases associated with asymmetric treatment modification that could favour one treatment over another.

Regulators might insist on data analysis by randomised treatment group (the more familiar approach used in traditional efficacy RCTs), but this could be misleading for a RRWE study. It was for this reason that safety data from the SLS were presented using both approaches: by randomised treatment group (strategy) and actual treatment (risk exposure). We hope that our observations will facilitate better understanding of RRWE studies as they increase in number.

New methods for evaluating benefit/risk are needed for RRWE studies in which treatment modifications are permitted and are often the norm. Furthermore, the importance of providing transparent reasoning and clear explanations for all key study design choices is emphasised by the inclusion of these points in the recommendations for prospective, observational, comparative effectiveness studies published by ISPOR [17].

The interpretation of safety data should be considered in the earliest stages of the design of RRWE studies and before preparation of the analysis plan. This is essential to ensure appropriate collection of data and its subsequent use. Unless safety data are handled correctly, a hazardous drug could potentially be mislabelled as safe or, alternatively, a safe drug blighted. A point also worth noting is that safety data from RRWE studies are unlikely to be suitable for integration into other types of analyses (e.g. meta-analyses, systematic reviews) due to inherent differences in trial design and the patient populations being studied.

As is the case for all interventional clinical studies, the SLS required expedited safety reporting of unexpected serious adverse reactions to regulatory authorities. Safety data from RRWE studies are also used in cumulative regulatory reporting through periodic benefit/risk evaluation reports, and as part of safety signal evaluation (actual treatment required). Safety data from RRWE studies will also be important for developing the safety profile of a drug, which requires safety data to be interpreted by actual treatment rather than the treatment strategy. If considered prior to study design, safety data from RRWE studies will also have value for evaluating benefit/risk in the setting of normal clinical practice.

Concluding remarks

RRWE studies provide the opportunity to evaluate medicines in patients and settings representative of routine clinical practice. The very elements that underpin the design of these studies can also impact significantly on the analysis, interpretation and implications of safety data collected in such trials. It is important to consider early in the design phase how best to evaluate safety elements in RRWE studies. As we have illustrated using data from the SLS, analysing safety by actual treatment received (i.e. accounting for the treatment modifications that occur in everyday primary care settings) is essential. While safety data analysis by randomisation group provides useful additional insights into treatment strategy, we propose that the use of both approaches could represent the ideal understanding of safety data from RRWE studies.

If healthcare professionals are to be fully informed about the safety of the medicines they are prescribing to patients and how these will perform in the everyday clinical setting, the evidence informing their decision-making must include safety data that has been captured, analysed and interpreted in a robust and reliable manner in real-world settings.

Acknowledgements

Editorial support during the development of the manuscript (in the form of writing assistance, assembling tables and figures, collating author comments, grammatical editing and referencing) was provided by Emma Landers at Ashfield MedComms (Macclesfield, UK), an Ashfield Health company, and was funded by GlaxoSmithKline plc.

Footnotes

  • This article has supplementary material available from openres.ersjournals.com

  • The Salford Lung Study (SLS) COPD and SLS Asthma are registered at www.clinicaltrials.gov with identifier numbers NCT01551758 and NCT01706198, respectively. Anonymised individual participant data from this study plus the annotated case report form, protocol, reporting and analysis plan, dataset specifications, raw dataset, analysis-ready dataset and clinical study report are available for research proposals approved by an independent review committee. Proposals should be submitted to www.clinicalstudydatarequest.com. A data access agreement will be required.

  • Author contributions: C. Harvey, Safety Development Leader (GlaxoSmithKline plc), led the manuscript writing together with Salford Lung Study (SLS) Steering/Publications Committee members (A. Woodcock, J. Vestbo, N.D. Bakerly and J.P. New) and the SLS study team (C. Crim, L. Frith, C. Williams, H. Elkhenini, N. Majeed, G. Cardwell, S. Collier, L. Jacques and J. Fletcher). The data reported, some of which have been published previously [9, 10], were derived from the SLS trial dataset. C. Harvey, A. Woodcock, J. Vestbo, C. Crim, L. Frith and J.P. New: study conception/design, data analysis/interpretation and manuscript writing/review. N.D. Bakerly and S. Collier: study conception/design, data acquisition, data analysis/interpretation and manuscript writing/review. C. Williams and J. Fletcher: study conception/design, data acquisition and manuscript writing/review. H. Elhkenini and N. Majeed: data acquisition and manuscript writing/review. G. Cardwell: data acquisition, data analysis/interpretation and manuscript writing/review. L. Jacques: data analysis/interpretation and manuscript writing/review. All authors approved the final version of the manuscript for submission. C. Harvey is the manuscript guarantor, and attests that all listed authors meet authorship criteria and that no other authors meeting the criteria have been omitted.

  • Conflict of interest: C. Harvey reports this study was funded by GlaxoSmithKline and medical writing support by Ashfield MedComms was also funded by GlaxoSmithKline; and she is an employee of and holds shares/options in GlaxoSmithKline.

  • Conflict of interest: A. Woodcock reports this study was funded by GlaxoSmithKline and medical writing support by Ashfield MedComms was also funded by GlaxoSmithKline; and a speaker fee and expenses ($3491.62) from GlaxoSmithKline, and advisory fees and expenses ($4998.95) from Chiesi, outside the submitted work. He is also chairman/shareholder of Reacta Biotech, a university spinout manufacturing food challenge materials for allergy.

  • Conflict of interest: J. Vestbo reports fees for attending six steering committee meetings over the study period from GlaxoSmithKline; and consultancy for COPD phase 2 and 3 programmes and payment for lectures including service in speaker bureaus from GlaxoSmithKline and Chiesi Pharmaceuticals, consultancy for COPD phase 2 and 3 programmes and payment for lectures including service in speaker bureau, and a biomarker study grant from Boehringer-Ingelheim, and consultancy for COPD phase 2 and 3 programes and payment for lectures including service in speaker bureaus from Novartis and AstraZeneca, outside the submitted work.

  • Conflict of interest: C. Crim reports reports this study was funded by GlaxoSmithKline and medical writing support by Ashfield MedComms was also funded by GlaxoSmithKline; and she is an employee of and holds shares/options in GlaxoSmithKline.

  • Conflict of interest: L. Frith reports reports this study was funded by GlaxoSmithKline and medical writing support by Ashfield MedComms was also funded by GlaxoSmithKline; and she is an employee of and holds shares/options in GlaxoSmithKline.

  • Conflict of interest: N.D. Bakerly reports reports this study was funded by GlaxoSmithKline and medical writing support by Ashfield MedComms was also funded by GlaxoSmithKline; and grants and personal fees from GlaxoSmithKline and Novartis, support for congress attendance from Boehringer Ingelheim, and grants and personal fees from Almirall/AstraZeneca, outside the submitted work.

  • Conflict of interest: J.P. New reports reports this study was funded by GlaxoSmithKline and medical writing support by Ashfield MedComms was also funded by GlaxoSmithKline.

  • Conflict of interest: C. Williams reports this study was funded by GlaxoSmithKline and medical writing support by Ashfield MedComms was also funded by GlaxoSmithKline; in addition, she was an employee of GlaxoSmithKline during the Salford Lung Studies.

  • Conflict of interest: H. Elkhenini reports this study was funded by GlaxoSmithKline and medical writing support by Ashfield MedComms was also funded by GlaxoSmithKline.

  • Conflict of interest: N. Majeed reports reports this study was funded by GlaxoSmithKline and medical writing support by Ashfield MedComms was also funded by GlaxoSmithKline.

  • Conflict of interest: G. Cardwell reports reports this study was funded by GlaxoSmithKline and medical writing support by Ashfield MedComms was also funded by GlaxoSmithKline; and he is an employee of and holds shares/options in GlaxoSmithKline.

  • Conflict of interest: S. Collier reports reports this study was funded by GlaxoSmithKline and medical writing support by Ashfield MedComms was also funded by GlaxoSmithKline; and she is an employee of and holds shares/options in GlaxoSmithKline.

  • Conflict of interest: L. Jacques reports reports this study was funded by GlaxoSmithKline and medical writing support by Ashfield MedComms was also funded by GlaxoSmithKline; and she is an employee of and holds shares/options in GlaxoSmithKline.

  • Conflict of interest: J. Fletcher reports reports this study was funded by GlaxoSmithKline and medical writing support by Ashfield MedComms was also funded by GlaxoSmithKline; and she is an employee of and holds shares/options in GlaxoSmithKline.

  • Support statement: The Salford Lung Study (SLS) COPD (HZC115151) and SLS Asthma (HZA115150) were funded by GlaxoSmithKline plc. The funder (GlaxoSmithKline plc) provided support in the form of salaries for authors C. Harvey, C. Crim, L. Frith, G. Cardwell, S. Collier, L. Jacques and J. Fletcher, and the specific roles of employees of the study funder in this work are detailed in the Author contributions section. J. Vestbo is supported by the National Institute for Health Research Manchester Biomedical Research Centre. Funding information for this article has been deposited with the Crossref Funder Registry.

  • Received December 21, 2020.
  • Accepted February 1, 2021.
  • Copyright ©The authors 2021
http://creativecommons.org/licenses/by-nc/4.0/

This version is distributed under the terms of the Creative Commons Attribution Non-Commercial Licence 4.0. For commercial reproduction rights and permissions contact permissions{at}ersnet.org

References

  1. ↵
    1. Global Initiative for Chronic Obstructive Lung Disease
    . Global strategy for the diagnosis, management, and prevention of chronic obstructive pulmonary disease (2020 report). https://goldcopd.org/wp-content/uploads/2019/11/GOLD-2020-REPORT-ver1.0wms.pdf Date last accessed: April 13, 2021.
  2. ↵
    1. Global Initiative for Asthma
    . Global strategy for asthma management and prevention (2019 update). https://ginasthma.org/wp-content/uploads/2019/06/GINA-2019-main-report-June-2019-wms.pdf Date last accessed: April 13, 2021.
  3. ↵
    1. Herland K,
    2. Akselsen JP,
    3. Skjønsberg OH, et al.
    How representative are clinical study patients with asthma or COPD for a larger “real life” population of patients with obstructive lung disease? Respir Med 2005; 99: 11–19. doi: 10.1016/j.rmed.2004.03.026
    OpenUrlCrossRefPubMed
  4. ↵
    1. Woodcock A,
    2. Boucot I,
    3. Leather DA, et al.
    Effectiveness versus efficacy trials in COPD: how study design influences outcomes and applicability. Eur Respir J 2018; 51: 1701531. doi: 10.1183/13993003.01531-201751
    OpenUrlAbstract/FREE Full Text
  5. ↵
    1. Chalkidou K,
    2. Tunis S,
    3. Whicher D, et al.
    The role for pragmatic randomized controlled trials (pRCTs) in comparative effectiveness research. Clin Trials 2012; 9: 436–446. doi: 10.1177/1740774512450097
    OpenUrlCrossRefPubMed
  6. ↵
    1. New JP,
    2. Bakerly ND,
    3. Leather D, et al.
    Obtaining real-world evidence: the Salford Lung Study. Thorax 2014; 69: 1152–1154. doi: 10.1136/thoraxjnl-2014-205259
    OpenUrlAbstract/FREE Full Text
    1. Bakerly ND,
    2. Woodcock A,
    3. New JP, et al.
    The Salford Lung Study protocol: a pragmatic, randomised phase III real-world effectiveness trial in chronic obstructive pulmonary disease. Respir Res 2015; 16: 101. doi: 10.1186/s12931-015-0267-6
    OpenUrlCrossRefPubMed
    1. Woodcock A,
    2. Bakerly ND,
    3. New JP, et al.
    The Salford Lung Study protocol: a pragmatic, randomised phase III real-world effectiveness trial in asthma. BMC Pulm Med 2015; 15: 160. doi: 10.1186/s12890-015-0150-8
    OpenUrlCrossRefPubMed
  7. ↵
    1. Vestbo J,
    2. Leather D,
    3. Diar Bakerly N, et al.
    Effectiveness of fluticasone furoate-vilanterol for COPD in clinical practice. N Engl J Med 2016; 375: 1253–1260. doi: 10.1056/NEJMoa1608033
    OpenUrl
  8. ↵
    1. Woodcock A,
    2. Vestbo J,
    3. Bakerly ND, et al.
    Effectiveness of fluticasone furoate plus vilanterol on asthma control in clinical practice: an open-label, parallel group, randomised controlled trial. Lancet 2017; 390: 2247–2255. doi: 10.1016/S0140-6736(17)32397-8
    OpenUrl
  9. ↵
    1. New JP,
    2. Leather D,
    3. Bakerly ND, et al.
    Putting patients in control of data from electronic health records. BMJ 2018; 360: j5554.
    OpenUrlCrossRefPubMed
  10. ↵
    1. Collier S,
    2. Harvey C,
    3. Brewster J, et al.
    Monitoring safety in a phase III real-world effectiveness trial: use of novel methodology in the Salford Lung Study. Pharmacoepidemiol Drug Saf 2017; 26: 344–352. doi: 10.1002/pds.4118
    OpenUrl
  11. ↵
    1. Thomas A,
    2. Lemanske RF Jr.,
    3. Jackson DJ
    . Approaches to stepping up and stepping down care in asthmatic patients. J Allergy Clin Immunol 2011; 128: 915–924. doi: 10.1016/j.jaci.2011.07.014
    OpenUrlCrossRefPubMed
  12. ↵
    1. Berlin JA,
    2. Glasser SC,
    3. Ellenberg SS
    . Adverse event detection in drug development: recommendations and obligations beyond phase 3. Am J Public Health 2008; 98: 1366–1371. doi: 10.2105/AJPH.2007.124537
    OpenUrlCrossRefPubMed
  13. ↵
    1. Clinical Study Report, HZC115151
    . A 12-month, open label, randomised, effectiveness study to evaluate fluticasone furoate (FF, GW685698)/vilanterol (VI, GW642444) inhalation powder delivered once daily via a novel dry powder inhaler (NDPI) compared with the existing COPD maintenance therapy alone in subjects with chronic obstructive pulmonary disease (COPD). GlaxoSmithKline, 2016. https://s3.amazonaws.com/ctr-gsk-7381/115151/f87a5b16-0c21-4f74-b440-777d991bc2c1/6f891b3b-dc6b-4982-b135-abab131f1cc0/gsk-115151-clinical-study-report-redact-v1.pdf Date last accessed: April 13, 2021.
  14. ↵
    1. Clinical Study Report, HZA115150
    . A 12-month, open label, randomised, effectiveness study to evaluate fluticasone furoate (FF, GW685698)/vilanterol (VI, GW642444) inhalation powder delivered once daily via a novel dry powder inhaler compared with usual maintenance therapy in subjects with asthma. GlaxoSmithKline, 2017. https://s3.amazonaws.com/ctr-gsk-7381/115150/6445927f-953c-402f-a79a-6ca22d93169c/e1e0596f-8dc8-4212-9e04-ce2a778d0e6d/gsk-115150-clinical-study-report-redact-v1.pdf Date last accessed: April 13, 2021.
  15. ↵
    1. Berger ML,
    2. Dreyer N,
    3. Anderson F, et al.
    Prospective observational studies to assess comparative effectiveness: the ISPOR good research practices task force report. Value Health 2012; 15: 217–230. doi: 10.1016/j.jval.2011.12.010
    OpenUrlCrossRefPubMed
PreviousNext
Back to top
Vol 7 Issue 2 Table of Contents
ERJ Open Research: 7 (2)
  • Table of Contents
  • Index by author
Email

Thank you for your interest in spreading the word on European Respiratory Society .

NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Enter multiple addresses on separate lines or separate them with commas.
Safety data in randomised real-world evidence studies: Salford Lung Study learnings
(Your Name) has sent you a message from European Respiratory Society
(Your Name) thought you would like to see the European Respiratory Society web site.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Print
Citation Tools
Safety data in randomised real-world evidence studies: Salford Lung Study learnings
Catherine Harvey, Ashley Woodcock, Jørgen Vestbo, Courtney Crim, Lucy Frith, Nawar Diar Bakerly, John P. New, Claire Williams, Hanaa Elkhenini, Nasir Majeed, Glenn Cardwell, Susan Collier, Loretta Jacques, Joanne Fletcher
ERJ Open Research Apr 2021, 7 (2) 00966-2020; DOI: 10.1183/23120541.00966-2020

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
Share
Safety data in randomised real-world evidence studies: Salford Lung Study learnings
Catherine Harvey, Ashley Woodcock, Jørgen Vestbo, Courtney Crim, Lucy Frith, Nawar Diar Bakerly, John P. New, Claire Williams, Hanaa Elkhenini, Nasir Majeed, Glenn Cardwell, Susan Collier, Loretta Jacques, Joanne Fletcher
ERJ Open Research Apr 2021, 7 (2) 00966-2020; DOI: 10.1183/23120541.00966-2020
del.icio.us logo Digg logo Reddit logo Technorati logo Twitter logo CiteULike logo Connotea logo Facebook logo Google logo Mendeley logo
Full Text (PDF)

Jump To

  • Article
    • Abstract
    • Abstract
    • Introduction
    • Safety data in the SLS
    • Understanding the impact of RRWE study design on safety data
    • Safety data in RRWE studies: examples from the SLS
    • Implications of safety data derived from RRWE studies
    • Concluding remarks
    • Acknowledgements
    • Footnotes
    • References
  • Figures & Data
  • Info & Metrics
  • PDF

Subjects

  • COPD and smoking
  • Asthma and allergy
  • Tweet Widget
  • Facebook Like
  • Google Plus One

More in this TOC Section

  • Type 2 inflammation in airway diseases
  • Telemedicine and virtual care in the era of COVID-19
  • BOS after lung or haematopoietic stem cell transplantation
Show more Reviews

Related Articles

Navigate

  • Home
  • Current issue
  • Archive

About ERJ Open Research

  • Editorial board
  • Journal information
  • Press
  • Permissions and reprints
  • Advertising

The European Respiratory Society

  • Society home
  • myERS
  • Privacy policy
  • Accessibility

ERS publications

  • European Respiratory Journal
  • ERJ Open Research
  • European Respiratory Review
  • Breathe
  • ERS books online
  • ERS Bookshop

Help

  • Feedback

For authors

  • Instructions for authors
  • Publication ethics and malpractice
  • Submit a manuscript

For readers

  • Alerts
  • Subjects
  • RSS

Subscriptions

  • Accessing the ERS publications

Contact us

European Respiratory Society
442 Glossop Road
Sheffield S10 2PX
United Kingdom
Tel: +44 114 2672860
Email: journals@ersnet.org

ISSN

Online ISSN: 2312-0541

Copyright © 2022 by the European Respiratory Society