National clinical audit for hospitalised exacerbations of COPD
- John R. Hurst1,2,3⇑,
- Jennifer K. Quint1,4,
- Robert A. Stone5,
- Yvonne Silove6,
- Jane Youde7 and
- C. Michael Roberts1,8
- 1National Asthma and COPD Audit Programme, Royal College of Physicians, London, UK
- 2UCL Respiratory, University College London, London, UK
- 3Royal Free London NHS Foundation Trust, London, UK
- 4National Heart and Lung Institute, Imperial College London, London, UK
- 5Somerset Lung Centre, Musgrove Park Hospital, Taunton, UK
- 6National Clinical Audit and Patient Outcomes Programme, Healthcare Quality Improvement Partnership, London, UK
- 7Care Quality Improvement Dept, Royal College of Physicians, London, UK
- 8School of Medicine and Dentistry, Queen Mary University of London, London, UK
- John R. Hurst, UCL Respiratory, London NW3 2PF, UK. E-mail: j.hurst{at}ucl.ac.uk
Abstract
Introduction Exacerbations of COPD requiring hospital admission are burdensome to patients and health services. Audit enables benchmarking performance between units and against national standards, and supports quality improvement. We summarise 23 years of UK audit for hospitalised COPD exacerbations to better understand which features of audit design have had most impact.
Method Pilot audits were performed in 1997 and 2001, with national cross-sectional audits in 2003, 2008 and 2014. Continuous audit commenced in 2017. Overall, 96% of eligible units took part in cross-sectional audit, 86% in the most recent round of continuous audit. We synthesised data from eight rounds of national COPD audit.
Results Clinical outcomes were observed to change at the same time as changes in delivery of care: length of stay halved from 8 to 4 days between 1997 and 2014, alongside wider availability of integrated care. Process indicators did not generally improve with sequential cross-sectional audit. Under continuous audit with quality improvement support, process indicators linked to financial incentives (early specialist review (55–66%) and provision of a discharge bundle (53–74%)) improved more rapidly than those not linked (availability of spirometry (40–46%) and timely noninvasive ventilation (21–24%)).
Conclusion Careful piloting and engagement can result in successful roll-out of cross-sectional national audit in a high-burden disease. Audit outcome measures and process indicators may be affected by changes in care pathways. Sequential cross-sectional national audit alone was not generally accompanied by improvements in care. However, improvements in process indicators were seen when continuous audit was combined with quality improvement support and, in particular, financial incentives.
Abstract
National audit in COPD is feasible and can improve process indicators, particularly when continuous audit is associated with quality improvement initiatives and financial incentives https://bit.ly/3eiOvOY
Introduction
COPD is a prevalent long-term respiratory condition with 174 million people affected globally [1]. In addition to a progressive decline in lung function accompanied by worsening symptoms and deteriorating functional performance, many people living with COPD experience acute deteriorations in respiratory health, termed “exacerbations” [2]. Exacerbations are important to patients [3] and health services (responsible for 50–75% of direct COPD healthcare costs [4]), especially those resulting in hospital admission, which contribute significantly to seasonal bed pressures.
There are a range of evidence-based interventions to treat exacerbations and reduce the risk of future events [2]. However, despite national and international guidelines and strategy documents addressing the management of COPD [5], there remains a translation gap [6] in which evidence from randomised-controlled trials (RCTs) is not evaluated in real-world settings, and where guideline recommendations are ineffectively implemented. Clinical audit is a key component of quality improvement (QI) programmes and a systematic review has suggested that clinical audit is associated with small but meaningful improvements in professional practice [7]. This review reported a 4.3% median improvement in compliance with desired practice, but a wide interquartile range (0.5%–16%), such that while a quarter of audits had larger benefits, a quarter were associated with no or negative impact [6]. The United Kingdom has a number of highly regarded national audits, introduced following the inclusion of audit as a pillar of clinical governance in the 1998 National Health Service (NHS) plan and with the explicit aim of QI. However, when subjected to rigorous evaluation, QI programmes have often had limited effect on outcome and process measures, particularly when the intervention is complex [8].
Over time, patterns of care evolve and audit outcomes and indicators may change with alterations in clinical pathways. Notably, for COPD (as in other ambulatory care-sensitive conditions [9]) there has been a move to treating people in their own homes rather than hospital [10], with increasing availability of admission avoidance and early supported discharge schemes.
There has been a UK national COPD audit programme addressing the organisation of care and management of COPD exacerbations for over 20 years. This has always included England and Wales, but not always Northern Ireland and Scotland (further detail of participation is included in the individual referenced audit reports, see below). The current UK National Asthma and COPD Audit Programme (NACAP) is commissioned by the Healthcare Quality Improvement Partnership (HQIP) as part of the National Clinical Audit and Patient Outcomes Programme in which UK hospitals are expected to participate. HQIP aims to promote QI in patient outcomes and in particular, to increase the impact that clinical audit, outcome review programmes and registries have on healthcare quality in England and Wales.
In this paper we describe more than 20 years of national cross-sectional, then continuous clinical audit for hospitalised COPD exacerbations in the United Kingdom to better understand which features of audit design have had most impact for patients, and to better inform those developing future large-scale audit programmes.
Methods
Three distinct national audit methodologies have been adopted: pilot (in which a limited number of hospitals took part, reporting in 1997 [11] and 2001 [12]), cross-sectional (open to all hospitals, recording a sample of cases over a defined period of time, reporting in 2003 [13], 2008 [14] and 2014 [15]) and from 2017, continuous audit (the latter intended to capture all eligible cases and with national reports provided annually and currently with three rounds of reporting) [16–18]. These clinical audit reports were supplemented with organisational reports published in 2003 [19], 2008 [20], 2014 [21] and 2018 [22], which provide complementary data on the numbers of admissions, staffing and pathways of care. A summary of the methodology underpinning each reporting cycle is included in table 1, with full details available in the referenced reports.
The first national audits, completed in 1997 and 2001 [11, 12], were designed to fact-find. These demonstrated variations in care between hospitals and a median standard of care that fell below recommendations [23]. While each participating unit was ultimately provided with its own performance, there was no formal QI support and neither were there incentives (financial or otherwise) to drive changes in care quality.
Overall 96% of UK trusts subsequently took part in a 2003 cross-sectional audit [13], which permitted the first national benchmarking of units and analysis of aspects of resource and organisation of care associated with clinical and process indicators. Thus, by careful roll-out, a national COPD audit was established and was seen to be valuable by clinicians and payers. Similar cross-sectional audits were conducted in 2008 [14] following publication of new National Institute for Health and Care Excellence (NICE) guidance [24] and again in 2014 [15] following publication of NICE quality standards [25].
From February 2017, COPD audit moved to continuous data collection in England and Wales. At the same time, two process indicators (review by a respiratory specialist within 24 h of admission, and provision of a “discharge bundle”) were linked to a financial incentive called the “Best Practice Tariff” for trusts in England (except those reimbursed by block contract). A standard British Thoracic Society (BTS) discharge bundle consists of support to quit smoking, referral to pulmonary rehabilitation, verification of inhaler technique, provision of written information for patients on COPD and arrangements for follow-up [26]. Units received complete payment when 60% or more of entered cases received both interventions. The continuous audit from 2017 was supported by a four-strand QI programme. This included provision of data to support local QI, such as run charts, available to sites in near real time; QI methodological support, including regional workshops teaching plan-do-see-act methodology [27], online case studies of best practice and provision of improvement tools, such as process charts and driver diagrams; robust patient and carer engagement and involvement and use of high-level change levers. For the latter, audit data were used to formulate NICE quality standards and in England, data were used by both the Care Quality Commission for hospital inspections and the Getting it Right First Time team (a national programme designed to improve medical care within the NHS by reducing unwarranted variation). Data were also used to inform the NHS England Long-Term Plan and to develop the aforementioned Best Practice Tariff. In Wales, audit data are included in the Annual Plan, and used in national care quality reporting and improvement mechanisms. Data from Scotland were first included under continuous audit in the 2018–2019 report.
The 30-day and 90-day readmission and mortality outcome data were obtained from the Office for National Statistics, Hospital-Episode Statistics and Patient Episode Database for Wales. Case ascertainment was calculated as the proportion of entered admissions to coded admissions. Outcomes were provided to commissioners and regulators, and made publicly available. Three rounds of reporting of continuous data have now been completed, most recently for people discharged up to September 2019 [16–18].
Audit data fields were periodically refined by expert committee based on the learning from previous audits, reflecting variables seen to be associated with unwarranted variation in care and/or which had a strong evidence base for association with patient outcomes. Fields were minimised to reduce burden on clinical teams, particularly for continuous audit and agreed by a Steering Group of stakeholders that latterly included patient and carer representatives.
Thus, through careful piloting and engagement of the target community, it was possible to successfully establish a cross-sectional national audit programme in a high-burden disease.
Having reviewed all the previous national COPD audit reports referenced above, key data were summarised in table 1. We provide a narrative synthesis of these data based on the trends observed. We interpret these trends in relation to changes occurring in the organisation of care.
Results
Characteristics of participating hospitals and patients
Selected data from respective rounds of audit are presented in table 1, divided into pilot, cross-sectional and continuous phases. In total these audits represent data from 225 463 hospital admissions for exacerbation of COPD.
Over 23 years of audit, the average age of people admitted has not varied by more than 1 year at around 72 years, but the sex ratio has reversed, such that by 2018–2019 there was a predominance of females (54% compared to 44% in 1997). Units have merged, such that it is difficult to assess changes in workload from these data. The median number of consultant respiratory physicians (median 5.3–7.5 whole time equivalents per 1000 admissions per unit), but not specialist nurses (1.9–1.3) or physiotherapists (2.2–1.9) increased between 2008 and 2017.
The latest available case ascertainment is 55%, with substantial variation between units; lower than in the final round of cross-sectional audit in 2014 at 67%. This may reflect problems with case identification or coding, but also the capacity of teams to complete continuous audit.
Change in length of stay, readmissions and mortality over time
Length of stay halved from a median of 8 days in 1997 to a median of 4 days from 2014, a change occurring at the same time as the wider availability of integrated care teams (45% in 2003 to 88% in 2017) and other drivers to reduce length of stay in ambulatory care-sensitive conditions. This association does not imply that the relationship is causal. There was a reduction in inpatient mortality from a peak of 7.8% in 2008 to 4.3% in 2014, occurring at the same time as an increase in the rate of readmissions from 33% to 43% and little change in 3-month mortality (14% and 12%, respectively). The latter may suggest post-discharge mortality (not directly measured) has increased.
Repeated cross-sectional audit alone was not associated with improved clinical outcomes
In general, outcomes that are not likely to be affected by changes in care pathways did not change through sequential rounds of cross-sectional audit. For example, the availability of spirometry (the diagnostic test that confirms a patient being treated for exacerbation has an underlying diagnosis of COPD) has not improved over time and is lower in the most recent round of continuous audit at 46% than the value of 55% obtained in the 2008 cross-sectional audit.
Taken together, these findings suggest that audit indicators may be affected by developments in clinical care pathways but, for COPD, sequential cross-sectional audit alone did not appear to be associated with significant improvements in care.
Continuous audit with financial incentives and QI support was associated with improved clinical outcomes
As illustrated in figure 1, the financially incentivised “best practice tariff” process indicator (comprising early specialist review and provision of a discharge bundle) improved during the continuous audit period. Delivery of the discharge bundle component increased from 53% in 2017 to 74% in 2018–2019.
In comparison, while the use of noninvasive ventilation (NIV) increased from 1997 with the increasing evidence basis for this intervention [28], timely application of NIV, which was not incentivised, has not improved over time (25% receiving this in under 3 h in 2008 compared to 24% receiving this in under 2 h in 2018–2019). As described above, the availability of spirometry results to confirm the diagnosis of COPD, also not incentivised, was lower in the 2018–2019 continuous audit at 46% than the peak of 55% under cross-sectional audit in 2008.
In summary, improvements in process indicators were rapid when continuous audit was combined with QI support and, in particular, when indicators were linked to financial incentives.
Discussion
We report a summary of more than 20 years' experience of UK national audit for hospitalised exacerbations of COPD. Our three key messages are first, that with careful piloting and engagement of the target community, it is possible to establish successfully a cross-sectional national audit programme in a high-burden disease. Second, audit indicators may be affected by developments in clinical care pathways but, for COPD, sequential cross-sectional audit alone does not in general appear to be associated with significant improvements in care. Finally, improvements in process indicators can be rapid when continuous audit is combined with QI support and, in particular, where those indicators are linked to financial incentives. Caution is required when interpreting temporal changes in performance metrics which are not necessarily causative or linked. However, our observation of improved care processes associated with application of the Best Practice Tariff indicator provides a model for increasing the impact of audit of hospitalised COPD exacerbations in other healthcare settings. These insights into the design features of continuous audits most likely to accelerate improvements in healthcare quality can also be applied to other high prevalence conditions in large health systems.
While more than 20 years of COPD audit has not been associated with improvements in clinical outcomes such as readmissions and mortality, the picture is much more nuanced and there are a number of important areas to consider. These include changes in care delivery and disease burden such as multimorbidity and admission acuity over time, the long lead time between interventions and impact on outcomes (e.g. smoking cessation) and the dependency of outcomes across multiple secondary, community and primary care providers. Audit data may be unable to demonstrate a relationship between care quality and clinical outcomes because it is not designed to do so in the same way as an RCT testing a single intervention under carefully controlled circumstances. Moreover, important and potentially more sensitive outcome measures such as quality of life have not been captured by the audit programme. These factors are considered in more detail in the sections below, necessary to inform our conclusion that there is indeed value in large-scale audit for high-burden conditions when this is provided with effective QI support.
COPD exacerbations are associated with high burden to patients and healthcare systems and there remains a translation gap getting evidence-based exacerbation-reduction interventions into daily clinical practice, despite the ready availability of guidelines. Clinical audit provides a tool to benchmark performance of individual units against each other and national standards, permitting changes to be made against which to assess improvements. The scale of benefit with improved performance is large, such that if those areas in England with a death rate above the median could achieve the median death rate for COPD, 3500 lives would be saved each year [29].
Care for long-term conditions, including COPD has evolved over time such that, for example, length of stay has reduced at the same time as the wider availability of supported discharge schemes and other “front door” initiatives. The decrease in length of stay we observed has occurred at the same time as a rise in readmissions, suggesting that a renewed focus on effective discharge and transition to community care is now necessary. It is challenging to interpret changes in outcomes over time in relation to audit strategy without also understanding changes in care delivery. Reduction in length of stay, but higher readmissions, occurred in the context of a decrease in inpatient mortality between 2008 and 2014 but little change in 30-day mortality. This suggests that out of hospital mortality at 30 days was increasing.
For most process indicators, sequential cross-sectional audit did not result in significant improvements in care. Barriers to improvement would include the timely provision of audit data back to clinical teams, with initial national reports published many months after data collection had been completed. Indeed, there was deteriorating performance on some basic process indicators such as the availability of a spirometry result to confirm a pre-existing diagnosis of COPD and poor implementation of key evidence-based interventions, including smoking cessation support and assessment for pulmonary rehabilitation. Cross-sectional audits were not accompanied by specific QI initiatives, or financial incentives. Building on and sharing best practice should reduce variation in care but was difficult to achieve and maintain. For this reason, from 2017, the UK national COPD audit programme began continuous national audit with investment in QI support (training, resources and real-time data feedback). This did result in improvements in process indicators of care quality, but primarily in those measures tied to financial incentives, in which case there could be rapid change. It is likely that QI support, financial incentives and real-time feedback of data are more important than cross-sectional versus continuous audit design. We are inferring improvement based on comparisons between historical data, and on the trend in variables over time. Notably, it is process indicators rather than clinical outcomes that have changed. Recent data in COPD [30] suggest that the simple existence of a care bundle is insufficient to drive improvements in clinical end points, such that complex interventions are only likely to work when effectively implemented.
There are similarities and differences between the COPD audit programme and other UK national audits. The UK National Hip Fracture Database recently reported that pay-for-performance was associated with reduced mortality, shorter length of stay and fewer readmissions by comparing performance in England (where care quality was financially incentivised) to Scotland (where it was not) over a period of 10 years [31]. Other UK national databases such as the Myocardial Ischaemia National Audit Project (MINAP) have demonstrated optimal use of guidelines to be associated with better outcomes after myocardial infarction [32], suggesting potential gains in COPD can be large if adherence to guidelines could be improved. National audit programmes present an opportunity for data collection at scale that can support and be supported by other national initiatives (e.g. the implementation of COPD care bundles is a current focus of work from the UK National Patient Safety Collaborative). In contrast to MINAP and the Hip Fracture Database, we have not seen reductions in mortality, likely due to the absence of definitive intervention for a COPD exacerbation analogous to coronary revascularisation or hip surgery with rehabilitation, and the long lead time to effects on mortality from interventions such as smoking cessation, critical in COPD and which is a component of the discharge bundle. Continuous audit for COPD is also a much more recent development compared, for example, to MINAP. Further time is required to assess whether improvement in COPD process indicators are associated with improvements in clinical outcomes such as readmissions and mortality and whether improvements in process indicators are maintained. However, it may also be the case that current outcome measures are insensitive to meaningful improvements in care.
While national systems such as the UK NHS lend themselves to formal audit, similar methodology could be applied in other types of health maintenance organisations or indeed smaller healthcare systems. We therefore provide a model of the implementation and routes to success for audit in a high-burden condition, in which audit at scales inspires and drives QI initiatives. Audit can also be useful internationally, as evidenced by the European COPD Audit, which reported that only the number of respiratory specialists per 1000 beds, an index of hospital resource, decreased the risk of post-discharge mortality [33]. Other aspects of COPD care such as the quality of outpatient provision may affect the risk of admission. Outpatient care is not part of the UK NACAP programme but has been conducted in Spain [34], demonstrating variable adherence to guidelines and quality standards. Audits in Spain have also highlighted the importance of understanding the effectiveness of QI [35], with recent large QI programmes in other disease areas not translating to improved clinical outcomes likely because the “dose” of QI was too small [8].
There are a number of challenges with continuous audit and for COPD in particular. These include burden on front-line staff, who need to be supported in viewing audit not as data collection for external use but as a tool for individual units to improve the quality of care they deliver. Teams will vary in their experience of and capacity for QI and may lack adequate administrative and technical support (such as bedside data capture) to complete audit effectively. There may be a disparity between actual practice and that recorded as being completed. Continuous national audit programmes are complex and expensive. There is a complex interplay between the organisation of care and reported audit metrics given that audit measures can become outdated as organisation of care changes and modifications to the organisation of care can itself be a response to audit results. This mandates periodic changes to the audit dataset. Case ascertainment remains variable and it is possible that the quality of care for people not entered into the audit may differ from those entered, and thus the best assessment of unit performance is to assess both outcomes and case ascertainment (considering outcomes in those not included in the audit). However, the relationship between case ascertainment and outcomes is complex, in that units with excellent performance may only need adopt a “sampling” approach to demonstrate continued excellence. There are arguments for and against sampling approaches, such as the audit of a selected proportion of admissions. Notably, we find that case ascertainment under continuous audit was less complete than with cross-sectional audits. Regarding COPD specifically, there is a fundamental problem with defining a COPD exacerbation, currently a clinical diagnosis of exclusion, and no robust method to adjust for case mix, given that the “severity” of a COPD exacerbation presentation is dependent on multiple factors, including the severity of the underlying COPD, of the exacerbation insult and the presence and severity of any comorbidities. Initial COPD exacerbation diagnoses may change and an initial alternative diagnosis may be amended to a diagnosis of COPD exacerbation complicating early review. Readmissions are a particularly challenging outcome to modify, reflecting care delivered during the index event and following discharge by primary and community teams, and to a certain extent part of the natural history of COPD with a documented increased risk of re-exacerbation in the period following recovery from a first exacerbation [36] and the presence of comorbidity [37]. Future audit might capture outcomes that are more sensitive to change, such as quality of life and patient experience measures.
We report temporal associations rather than causation between continuous audit, QI support, financial incentives and improved process indicators. In addition, it is challenging to know which component of the audit has been most successful, though it is notable that those indicators linked to financial incentives improved more than indicators that were not. A general limitation of clinical audit is that while audit can demonstrate changes in care quality process items, it is not designed to demonstrate causal links with clinical outcomes as would be the case in a RCT. This emphasises the need to align guidelines and audit to real-world clinical trials. A systematic review has reported that audit generally results in only small improvements in practice, with effectiveness dependent on both baseline performance and the way that feedback is provided [7]. We demonstrate larger improvements when continuous audit was delivered in association with QI support and financial incentives.
A limitation of this report is that, in line with original permissions granted, source data were destroyed after analysis. This has prevented new analysis of the data such that we present a narrative synthesis of findings over time, rather than adopting an experimental approach
Clinical outcomes for chronic diseases such as COPD are dependent on multiple providers within a health service. The importance of this is emphasised by the need to better understand the reduction in inpatient mortality while 3-month mortality did not change. NACAP is already linking inpatient care with subsequent outcomes such as mortality and readmissions. The ultimate aim will be to link individual patients' hospital admission data with the primary care and pulmonary rehabilitation work streams that are also part of NACAP. There are considerable challenges to achieving this, but also increasing opportunities to do so through better linkage across different electronic health record systems. We recognise that a proportion of COPD care in the United Kingdom is also delivered by multi-professional community-based services that are not, at present, audited through NACAP.
In conclusion, it is feasible to establish and maintain large-scale audit programmes for a high-burden disease such as COPD and continuous national audit with explicit QI support and high-level change levers, including financial incentives, has been temporally associated with rapid, clinically meaningful improvement in process indicators for people admitted to hospital with COPD exacerbations This has, so far, not translated into improvement in hard clinical outcomes such as mortality and readmission rate. NACAP provides a model for audit-driven improvement of COPD care in other healthcare settings and for national audit of other high-burden conditions.
Footnotes
Conflict of interest: J.R. Hurst reports a grant to UCL, fees to himself and UCL for educational and advisory work, and support to attend meetings from pharmaceutical companies that make medicines to treat COPD, outside the submitted work.
Conflict of interest: J.K. Quint reports grants from Royal College of Physicians during the conduct of the study (she is the analysis lead for NACAP); and grants from the MRC, the BLF, The Health Foundation and Asthma UK, grants and personal fees from AZ, Bayer and BI, grants from Chiesi, and grants and personal fees from GSK, outside the submitted work.
Conflict of interest: A. Stone has nothing to disclose.
Conflict of interest: Y. Silove reports in her role as Associate Director for the National Clinical Audit and Patient Outcomes Programme at HQIP, she is the sponsor for the NACAP contract, which HQIP commissions on behalf of NHS England, the Welsh Government and the Scottish Government.
Conflict of interest: J. Youde reports this audit is commissioned by HQIP for delivery through the Royal College of Physicians London (RCP) and her employing trust is paid for her time at the RCP.
Conflict of interest: C.M. Roberts has nothing to disclose.
- Received April 23, 2020.
- Accepted June 10, 2020.
- Copyright ©ERS 2020
This article is open access and distributed under the terms of the Creative Commons Attribution Non-Commercial Licence 4.0.