RT Journal Article
SR Electronic
T1 AI-based software facilitates spirometry quality control in asthma and COPD clinical trials
JF ERJ Open Research
JO erjor
FD European Respiratory Society
SP 00292-2022
DO 10.1183/23120541.00292-2022
A1 Topole, Eva
A1 Biondaro, Sonia
A1 Montagna, Isabella
A1 Corre, Sandrine
A1 Corradi, Massimo
A1 Stanojevic, Sanja
A1 Graham, Brian
A1 Das, Nilakash
A1 Ray, Kevin
A1 Topalovic, Marko
YR 2022
UL http://openres.ersjournals.com/content/early/2022/09/22/23120541.00292-2022.abstract
AB Rationale Acquiring high-quality spirometry data in clinical trials is important, particularly when using FEV1 or FVC as primary endpoints. In addition to quantitative criteria, the ATS/ERS standards include subjective evaluation which introduces inter-rater variability and potential mistakes. We explored the value of AI-based software (ArtiQ.QC) to assess spirometry quality and compared it to traditional over-reading control.Methods A random sample of 2000 sessions (8258 curves) was selected from Chiesi COPD and Asthma trials (N=1000 per disease). Acceptability using the 2005 ATS/ERS standards was determined by over-reader review and by ArtiQ.QC. Additionally, three respiratory physicians jointly reviewed a subset of curves (N=150).Results The majority of curves (N=7267, 88%) were of good quality. The AI agreed with over-readers in 91% of cases, with 97% sensitivity and 93% positive predictive value. Performance was significantly better in the asthma group. In the revised subset, N=50 curves were repeated to assess intra-rater reliability, (Kappa: 0.83, 0.86 and 0.80). All reviewers agreed on 63% of 100 unique tests (Kappa=0.5). When reviewers set the consensus (gold-standard), individual agreement with it was 88%, 94% and 70%. The agreement between AI and “gold-standard” was 73%, over reader agreement was 46%.Conclusion AI-based software can be used to measure spirometry data quality with comparable accuracy as experts. The assessment is a subjective exercise, with intra- and inter-rater variability even when the criteria are defined very precisely and objectively. By providing consistent results and immediate feedback to the sites, AI may benefit clinical trial conduct and variability reduction.FootnotesThis manuscript has recently been accepted for publication in the ERJ Open Research. It is published here in its accepted form prior to copyediting and typesetting by our production team. After these production processes are complete and the authors have approved the resulting proofs, the article will move to the latest issue of the ERJOR online. Please open or download the PDF to view this article.Conflict of interest: Sandrine CORRE received support for the present manuscript from Chiesi, for which they are an employee.Conflict of interest: Marko Topalovic received support for the present manuscript from ArtiQ, for which they are a co-founder.Conflict of interest: Eva Topole received support for the present manuscript from Chiesi, for which they are an employee.Conflict of interest: Isabella Montagna received support for the present manuscript from Chiesi, for which they are an employee.Conflict of interest: BIONDARO Sonia received support for the present manuscript from Chiesi, for which they are an employee.Conflict of interest: Sanja Stanojevic has received consulting fees from Chiesi Farmaceuticals, outside the submitted work.Conflict of interest: Brian L Graham reports the following relationships outside the submitted work; received professional fees from Chiesi Farmaceutici S.p.A., MGC Diagnostics, Vyaire Medical, and the Lung Association of Saskatchewan. BG has a patent licensing agreement with Hans Rudolph Inc.Conflict of interest: Kevin Ray received support for the present manuscript from ArtiQ, for which they are an employee.Conflict of interest: NILAKASH DAS received support for the present manuscript from KU Leuven university for which they were a Post-doctoral researcher; patent pending for ArtiQ; stock options in lieu of patents held for ArtiQ.Conflict of interest: Massimo Corradi has a patent pending.