Abstract
Introduction Exhaled-breath analysis of volatile organic compounds could detect lung cancer earlier, possibly leading to improved outcomes. Combining exhaled-breath data with clinical parameters may improve lung cancer diagnosis.
Methods Based on data from a previous multi-centre study, this article reports additional analyses. 138 subjects with non-small cell lung cancer (NSCLC) and 143 controls without NSCLC breathed into the Aeonose. The diagnostic accuracy, presented as area under the receiver operating characteristic curve (AUC-ROC), of the Aeonose itself was compared with 1) performing a multivariate logistic regression analysis of the distinct clinical parameters obtained, and 2) using this clinical information beforehand in the training process of the artificial neural network (ANN) for the breath analysis.
Results NSCLC patients (mean±sd age 67.1±9.1 years, 58% male) were compared with controls (62.1±7.0 years, 40.6% male). The AUC-ROC of the classification value of the Aeonose itself was 0.75 (95% CI 0.69–0.81). Adding age, number of pack-years and presence of COPD to this value in a multivariate regression analysis resulted in an improved performance with an AUC-ROC of 0.86 (95% CI 0.81–0.90). Adding these clinical variables beforehand to the ANN for classifying the breath print also led to an improved performance with an AUC-ROC of 0.84 (95% CI 0.79–0.89).
Conclusions Adding readily available clinical information to the classification value of exhaled-breath analysis with the Aeonose, either post hoc in a multivariate regression analysis or a priori to the ANN, significantly improves the diagnostic accuracy to detect the presence or absence of lung cancer.
Abstract
Adding readily available clinical information to the classification value of exhaled-breath analysis with the Aeonose significantly improves the diagnostic accuracy to detect the presence or absence of lung cancer http://bit.ly/38ps6fH
Introduction
Lung cancer remains the leading cause of cancer-related death worldwide, accounting for ∼5% of total mortality in many countries [1]. Unfortunately, most patients present with late-stage disease (stages III and IV) and there are limited effective treatment options with consequently high mortality rates (5-year survival rate <10%) [2, 3]. Currently, the only screening method leading to reduced lung cancer mortality in high-risk groups is annual low-dose computed tomography (LDCT) [4, 5]. However, LDCT screening for lung cancer has also resulted in a notable rate of false-positive cases, leading to unnecessary invasive procedures, risks due to radiation exposure and unnecessary anxiety [4–8]. In Europe, results of the Dutch–Belgian lung cancer screening trial (NELSON) are awaited before a decision on implementation of screening programmes in Europe will be made [9]. Hence, there is an increasing demand for innovative, noninvasive, point-of-care diagnostic tools to detect lung cancer at an early stage.
Exhaled-breath analysis with electronic nose technology is a technique based on detecting combinations of volatile organic compounds (VOCs) that are exhaled in very low concentrations. These VOCs reflect pathophysiological processes, such as infection, inflammation and neoplasms [10–12]. VOCs are of interest since they might be directly related to the presence of diseases, they can be tested noninvasively and pattern recognition techniques can serve as classifiers for diseases. Several studies on exhaled-breath analysis have supported the hypothesis that VOC patterns alter when lung cancer is present [13–20].
Recently, we reported results of a study (including 290 subjects) differentiating subjects with lung cancer, including classification into subtypes of lung cancer, from healthy individuals by means of exhaled-breath analysis with the Aeonose (The eNose Company, Zutphen, the Netherlands) [13]. In this study, an artificial neural network (ANN) was trained using exhaled-breath data only.
The Aeonose was able to diagnose patients with non-small cell lung cancer (NSCLC) with a sensitivity of 94%, a negative predictive value (NPV) of 85% and an area under the receiver operating characteristic curve (AUC-ROC) of 0.76. Subtyping NSCLC into adenocarcinoma and squamous cell carcinoma also showed promising results.
These diagnostic parameters were based on the analysis of exhaled VOCs only and did not take into account any of the subjects' risk factors, such as age, sex, smoking status (number of pack-years) and presence of COPD. This paper describes the potential of adding specific clinical information to the classification value obtained from the Aeonose on the diagnostic accuracy to diagnose lung cancer. The hypothesis was that adding clinical information would improve the diagnostic performance. This was assessed in two ways: first, the clinical information was added afterwards to the classification value of the Aeonose as obtained from the ANN by applying multivariate logistic regression analysis, and second, by using this clinical information a priori in the training process of the ANN.
Methods
Data were obtained from a previous prospective, multi-centre study where subjects suspected for lung cancer, as well as healthy volunteers, were asked to participate [13]. The originally collected breath samples were currently used for additional analyses. The four secondary teaching hospitals participating were Medisch Spectrum Twente (Enschede, the Netherlands), Bernhoven (Uden, the Netherlands), Medisch Centrum Leeuwarden (Leeuwarden, the Netherlands) and Deventer Ziekenhuis (Deventer, the Netherlands). For patients with confirmed lung cancer based on histopathology, staging was established according to the seventh edition of the American Joint Committee on Cancer TNM (tumour, node, metastasis) staging system [21]. The control group consisted of suspected subjects with a rejected diagnosis based on imaging and/or derived histopathology and healthy volunteers. Healthy volunteers with a minimum age of 50 years were recruited through an advertisement on the hospitals' websites. There were two exclusion criteria for all subjects: another active malignancy in the past 5 years or the inability to perform a complete Aeonose measurement. Demographic data were collected including age, sex, body mass index, smoking status, number of pack-years, and presence of COPD, hypertension and diabetes mellitus.
The Aeonose is a handheld electronic nose device containing three metal-oxide sensors [13, 22, 23]. This device is a noninvasive, easy-to-use, low-cost tool that is, once trained and validated, able to perform real-time analysis to detect lung cancer. Temporary storage of the breath sample is not required. Subjects were instructed to breathe through the Aeonose for 5 minutes with their nose clipped to prevent nose breathing.
The study protocol was approved by the medical ethics committee of Medisch Spectrum Twente and by the board of directors at each participating centre. All subjects gave informed consent.
Statistical analysis
Continuous variables are reported as mean with corresponding standard deviation or as median with interquartile range. Nominal variables are reported as numbers with corresponding percentages. To assess differences between the groups, either the t-test for normally distributed continuous variables, Mann–Whitney U-test for skewed distributed continuous or ordinal variables, or Chi-squared test for nominal and categorical variables was applied. Number of pack-years was categorised as 0, 1–20, 21–40 and >40 pack-years. Based on clinical reasoning, we assumed a strong relationship between smoking status and number of pack-years, which was confirmed (p<0.001). Number of pack-years contained most relevant information. Therefore, we excluded smoking status as clinical variable from the multivariate analysis.
Data from exhaled breath were analysed using Aethena, a proprietary, dedicated software package from The eNose Company. The software package comprises techniques for data pre-treatment, data compression methods, ANN training and classification to assess the probability of lung cancer, ranging a single value between −1 and 1 [23]. ANNs have been developed as an alternative statistical technique to perform multifactorial analyses by interconnecting nodes with weighted connection lines to predict outcomes or classifying values on an individualised basis [24]. Sensitivity, specificity, positive predictive value (PPV) and NPV were calculated for the diagnosis of lung cancer based on the classification by the Aeonose, and receiver operating characteristics (ROC) curves were composed with a corresponding AUC-ROC with 95% confidence interval.
Clinical variables that were univariately associated with the presence of lung cancer (p<0.15) were entered in a multivariate logistic regression analysis where variables with the highest p-values were eliminated step-by-step (backward method) until the fit of the model decreased significantly, based on the −2 log likelihood. This analysis was based on clinical variables only.
Subsequently, two types of multivariate analysis were performed where breath data were included. First, another multivariate logistic regression analysis, consistent with the aforementioned method, was performed together with the classification value of the Aeonose as obtained from the ANN. Second, clinical variables that were univariately associated with the presence of lung cancer (p<0.15) were added to the vector containing breath profile information once data compression had been realised. These extended vectors (one per subject) were used for training the ANN.
Sensitivity, specificity, positive predictive value (PPV), NPV and AUC-ROC were then calculated for the diagnosis of lung cancer according to the selected multivariate logistic regression model and the extended ANN. These outcomes were compared with the diagnostic accuracy obtained by the classification result of the exhaled-breath analysis only. The multivariate regression model was internally validated by 1000 iterations of bootstrap.
All statistical tests were two-sided with a significance level at 0.05. SPSS version 24.0 (IBM, Armonk, NY, USA) was used to perform statistical mathematics.
Results
A total of 281 subjects were included, of whom 138 had confirmed NSCLC. The control group consisted of 143 subjects without lung cancer of whom 59 were suspected for lung cancer but were considered negative after investigation, and 84 subjects were healthy volunteers (figure 1). Table 1 provides a description of the study participants including clinical characteristics for both groups.
Flow chart showing the different groups. NSCLC: non-small cell lung cancer.
Clinical characteristics of subjects
Lung cancer patients were significantly older (mean age 67.1±9.1 years), more likely to be male and a current or ex-smoker, had smoked more pack-years, and were more often diagnosed with COPD than subjects in the control group. Almost 75% of the NSCLC patients were classified as stage III or IV disease.
Table 2 shows that sex, age, smoking status, number of pack-years, presence of COPD and the classification value obtained by the Aeonose were univariately associated with the presence of lung cancer. Subsequently, we added these candidate variables to a multivariate regression analysis, which showed that age, number of pack-years, presence of COPD and the value of the Aeonose remained significantly predictive for the presence of lung cancer.
Results of the univariate and multivariate logistic regression analyses for diagnosing lung cancer
Each additional year of age was associated with a 5% higher chance of having lung cancer (OR 1.05, 95% CI 1.02–1.09). Subjects who had smoked 1–20 pack-years have a 3.5-fold higher chance of developing lung cancer (OR 3.5, 95% CI 1.25–9.66), whereas those who had smoked >40 pack-years have a 11.7-fold higher chance (OR 11.7, 95% CI 4.04–33.87). Patients with COPD had a 2.3-fold increased risk of having lung cancer (OR 2.27, 95% CI 1.18–4.43). The classification value of the Aeonose was also strongly associated with the presence of lung cancer (OR 12.7, 95% CI 4.48–35.83).
The multivariate logistic regression analysis based on clinical variables only showed a sensitivity of 93.5%, a specificity of 50%, a PPV of 64.5% and an NPV of 88.8%. This corresponded with an AUC-ROC of 0.80 (95% CI 0.75–0.85).
When the ANN was trained with exhaled-breath data from the Aeonose only, we found a sensitivity of 94.2%, a specificity of 44.1%, and a PPV and NPV of 61.9% and 88.7%, respectively, with an AUC-ROC of 0.75 (95% CI 0.69–0.81) (table 3). When applying the multivariate logistic regression model including the resulting value (−1 to 1) of the exhaled-breath data from the ANN in the exact same study population, we found an improved performance to distinguish NSCLC patients from controls with an AUC-ROC of 0.86 (95% CI 0.81–0.90). By choosing a relevant threshold value in the ROC curve, focusing on high sensitivity and high NPV, the analysis showed a sensitivity of 95.7%, a specificity of 59.7%, and a PPV and NPV of 69.5% and 92.5%, respectively. The bootstrap analysis for internal validation showed similar regression coefficients compared to our original model showing robustness of the model.
Diagnostic performance of the three investigated prediction models
When training the ANN with exhaled-breath data together with the clinical variables that were univariately associated with the presence of lung cancer, we found an improved diagnostic performance as well to distinguish NSCLC patients from controls, showing an AUC-ROC of 0.84 (95% CI 0.79–0.89). By choosing an appropriate threshold value in the ROC diagram, we observed a sensitivity of 94.2%, a specificity of 49.0%, and a PPV and NPV of 64.0% and 89.7% respectively. Figure 2 shows the combined ROC curve showing the improved performance of both multivariate models.
Combined receiver operating characteristic curve showing four predictive algorithms: logistic regression clinical variables, single Aeonose value, extended artificial neural network (ANN) and logistic regression including Aeonose value.
Discussion
This study assessed the impact of combining exhaled-breath analysis and clinical parameters in diagnosing lung cancer. We showed that adding readily available clinical information to the classification value of exhaled-breath analysis by the Aeonose in a relatively easy-to-perform multivariate regression model improved the diagnostic accuracy, expressed as AUC-ROC, from 0.75 to 0.86 to diagnose lung cancer in a noninvasive way. When extending the vector containing compressed breath data with clinical variables, and in this way training the ANN to distinguish between sick and healthy individuals, the diagnostic accuracy, expressed as AUC-ROC, increased from 0.75 to 0.84. Minor differences were observed compared to our previous performed analysis due to the somewhat smaller sample size because of missing information on pack-years (previous study: n=290, AUC-ROC 0.76) and the fact that outcomes of frequently training an ANN can slightly fluctuate.
It turns out that logistic regression analysis and ANN are equally capable of increasing classification quality of lung cancer diagnosis. We expected to see rather improved accuracy when entering a combination of clinical and exhaled-breath data directly to the untrained ANN, because it considers possible interactions. Independency of parameters, like breath profile, COPD and pack-years, cannot be fully assumed so far. As extending exhaled-breath data with clinical parameters followed by classification by an ANN is more complex than building a multivariate regression model out of single clinical parameters, the latter is to be recommended for practical use.
Besides ANN and logistic regression analysis, other statistical learning methods for classifying exhaled-breath data are available, such as random forest and support vector machines [25]. In a previous study, results from neural network analysis were compared to results obtained from random forest and support vector machine showing comparable diagnostic performance [26]. In this study, we only focused on the two statistical learning methods described: logistic regression analysis and neural network analysis.
Previous studies have shown that electronic nose technology based on pattern recognition of VOCs or identifying VOCs with gas chromatography/mass spectrometry can differentiate between subjects with and without lung cancer [13, 14, 17, 20, 27–29]. Several studies using techniques for VOC identification have used logistic regression analysis to identify lung cancer-specific VOCs [30, 31]. However, logistic regression analysis including clinical parameters in studies using pattern recognition techniques has not been shown often yet. Tirzïte et al. [19] used logistic regression analysis to predict the presence of lung cancer with the Cyranose 320 electronic nose mainly using segments of exhaled breath as input variables for the logistic regression analysis, but also including a few clinical parameters, such as age, smoking status, smoking history and ambient temperature. They were able to distinguish subjects with lung cancer from controls with a sensitivity of 96% in both smokers and nonsmokers, and a specificity >90% in both groups. To our knowledge, no studies have been performed using clinical variables and exhaled-breath data based on pattern recognition combined in an ANN to diagnose lung cancer. However, several studies have used ANNs to detect lung cancer without performing exhaled-breath analysis. These studies mainly focused on clinical parameters and biomarkers based on blood and genetic abnormalities [32, 33]. As described in our training study, the optimal cut-off point chosen determines the number of false-positive cases versus the number of missed cases concerning lung cancer [13]. We focused again on a high sensitivity and a high NPV, since lung cancer has an extremely high mortality rate if not detected early. By adding the clinical variables to the exhaled-breath data, we saw in both models that all diagnostic parameters improved, thereby reaching higher sensitivity and NPV compared to the training study, but we also observed fewer false-positive cases by achieving higher specificity.
In the near future, the results obtained could be proposed to add value in several ways. First, in the case of implementation of LDCT in Europe, the Aeonose may be deployed after suspicion of lung cancer has been raised with LDCT. Due to the high NPV with the Aeonose, subjects could be prevented from undergoing unnecessary invasive interventions and be monitored with prolonged intervals [8]. In addition, there is current debate about identifying at-risk groups relevant for LDCT screening [34–36]. Combining clinical parameters and exhaled-breath data in an ANN could indicate the degree of suspicion of lung cancer, and therefore serve as an adjunct for risk stratification in lung cancer screening, supporting clinical decision making.
Limitations of our study should also be mentioned. We did not analyse subjects with small cell lung cancer, nor did we analyse differences between the histological subtypes of NSCLC and lung cancer stages, since these subgroups were too small to include all relevant variables in the multivariate model. Moreover, we do not know the influence of food intake by subjects. Eating and drinking was not restricted before the exhaled-breath measurement. However, the neural network is being trained comparing breath profiles of positive and negative subjects regardless their food intake. When numbers of subjects are sufficiently large, it can be assumed that food intake is not relevant as it averages out. It should also be noted that most subjects had stage III and IV lung cancer (75%). This population differs from the high-risk asymptomatic subjects suitable for screening, where focus lies in early-stage lung cancer. However, the risk factors included in the multivariate analysis are applicable for both early- and late-stage lung cancer, so when exhaled-breath analysis is able to detect early-stage lung cancer, readily available clinical information should be incorporated in the analysis. Future analysis, including sufficient stage I and II NSCLC, should indicate whether breath patterns already change early in the course of the disease.
We should also note that in high-dimensional data sets as obtained with the Aeonose, the problem of overfitting can occur where a prediction model that looks appropriate on training data used to develop it, will perform poorly on future observations. Combining analytical techniques, such as data compression and cross-validation, partly overcomes this issue. Currently, an external validation study is performed where a complete new cohort of subjects is included to totally overcome the issue of overfitting.
Conclusion
Due to the aggressive nature of lung cancer, diagnostic accuracy should be as high as possible. This diagnostic accuracy to detect the presence or absence of lung cancer by exhaled-breath analysis with the Aeonose can be improved by adding readily available clinical information, either post hoc in a multivariate logistic regression model or a priori in the training process to the ANN, compared to the single classification value based on exhaled-breath data only. As both approaches yielded similar results, the multivariate logistic regression model should be preferred as its application is more convenient.
Footnotes
Conflict of interest: S. Kort reports an unrestricted research grant paid to her institution by The eNose Company, Zutphen, during the conduct of the study.
Conflict of interest: M. Brusse-Keizer has nothing to disclose.
Conflict of interest: J.W. Gerritsen is an employee of The eNose Company, Zutphen.
Conflict of interest: J.H. Schouwink has nothing to disclose.
Conflict of interest: E. Citgez has nothing to disclose.
Conflict of interest: F.H.C. de Jongh has nothing to disclose.
Conflict of interest: J. van der Maten has nothing to disclose.
Conflict of interest: S. Samii has nothing to disclose.
Conflict of interest: M. van den Bogart has nothing to disclose.
Conflict of interest: J. van der Palen has nothing to disclose.
- Received August 28, 2019.
- Accepted January 14, 2020.
- Copyright ©ERS 2020
This article is open access and distributed under the terms of the Creative Commons Attribution Non-Commercial Licence 4.0.
References
