Abstract
Background American Thoracic Society/European Respiratory Society guidelines recommend context-specific exposure assessments to diagnose interstitial lung disease (ILD). In sub-Saharan Africa, ILD diagnoses are rare, and locally validated ILD exposure questionnaires are not used.
Methods A physician-administered ILD exposure questionnaire was developed using a four-step mixed-methods modified Delphi approach. First, ILD questionnaires from high-income countries and data from Pneumotox were reviewed, compiled and face-validated. Second, a local pilot group of ILD experts ranked item relevance using a Likert scale and suggested additions. Third, the questionnaire format and pilot rankings were addressed in a focus group discussion that was analysed using grounded theory. Finally, following focus group discussion modifications, the resulting items (with three duplicate item groups for evaluation of internal consistency) were ranked for importance by members of the Pan-African Thoracic Society (PATS).
Results Face validation resulted in 82 items in four categories: “Smoking and Drugs”, “Environmental Exposures”, “Occupations” and “Medications”. Pilot group (n=10) ranking revealed 27 outliers and 30 novel suggestions. Focus group (n=12) discussion resulted in 10 item deletions, 14 additions and 22 re-wordings; themes included desire for extensive questionnaires and stigma sensitivity. Final validation involved 58 PATS members (mean±sd age 46±10.6 years, 76% male, from 17 countries) ranking 84 items derived from previous steps and three duplicate question groups. The questionnaire was internally consistent (Cronbach's α >0.80) and ultimately included 73 items.
Conclusion This mixed-methods study included experts from 17 countries in sub-Saharan Africa and successfully developed a 73-item ILD exposure questionnaire for sub-Saharan Africa. African pulmonary experts valued region-specific additions and ranked several items from existing ILD questionnaires as unimportant.
Abstract
Guidelines for ILD diagnosis recommend locally relevant exposure questionnaires. An ILD exposure questionnaire has been developed and validated for sub-Saharan Africa to provide a locally relevant questionnaire for this region. https://bit.ly/3ObwaFg
Introduction
Interstitial lung disease (ILD) encompasses a group of heterogeneous disorders with over 200 separate entities. While classification schemes for ILD vary, they are often separated into three categories: ILD related to underlying diseases, such as connective tissue disease; ILD related to medications or environmental exposure, such as hypersensitivity pneumonitis or pneumoconiosis; and idiopathic ILDs, such as idiopathic pulmonary fibrosis or sarcoidosis [1]. Within low- and middle-income countries, and specifically sub-Saharan Africa (sSA), there has been limited ILD research. However, environmental risk factors for ILD are common in sSA and population-based pulmonary function studies have shown high rates of preserved ratio impaired spirometry, raising the possibility that ILD diagnosis may frequently be missed [2–4].
Treatment and prognosis of ILD is dependent on aetiology. Current guidelines recommend a thorough review of exposures when evaluating an individual with ILD [1]. Despite this recommendation, ILD is often misclassified, and relevant exposures are overlooked. In one study, 46% of patients referred to a tertiary ILD clinic for idiopathic pulmonary fibrosis were ultimately diagnosed with hypersensitivity pneumonitis [5]. In sSA, there are barriers to guideline-based diagnostic workup for ILD that include lack of knowledge about ILD, limited access to computed tomography scanners and pulmonary function testing, and a shortage of multidisciplinary expert panels for ILD diagnosis [3, 6].
Questionnaires are useful clinical tools and have demonstrated benefits in various settings; they can increase pulmonologist adherence to guidelines and improve case finding for respiratory diseases, such as chronic obstructive pulmonary disease or obstructive sleep apnoea [7]. Furthermore, questionnaires can identify pertinent occupational exposures and the American Thoracic Society/European Respiratory Society recommends using questionnaires to evaluate patients with suspected ILD [1, 8]. Many clinics in high-income countries use exposure questionnaires along with radiological, clinical and laboratory evaluation to accurately diagnose ILD; unfortunately, there are no instruments, to our knowledge, that screen for exposures in sSA [1, 6].
The Delphi method is a well-established technique for consensus gathering within medical research. It includes literature review, opinion of stakeholders and the judgment of experts within a field to reach agreement [9]. The modified Delphi method for questionnaire development comprises four stages: identification of a research problem, a literature search, development of a questionnaire by a panel of experts and, finally, an iterative process in which the modified questionnaire is sent to an external expert panel for anonymous serial ranking of each question or component (generally two or three cycles) [9, 10]. There are several accepted modifications to the original Delphi protocol; one modification involves inclusion of face-to-face or focus group discussions during the iterative Delphi process and is termed a qualitative Delphi protocol [11].
Given the benefit of questionnaires for identifying exposures in patients with suspected ILD and the challenges with ILD diagnosis in sSA, we developed a healthcare worker-administered exposure questionnaire for patients with ILD using mixed-methods and a qualitative Delphi approach.
Methods
We developed the sSA ILD exposure questionnaire using a four-step mixed-method technique based on a qualitative Delphi protocol, including 1) background research and face validation, 2) questionnaire item-ranking on a Likert scale, 3) focus group discussion of the initial Likert ranking and 4) Likert ranking by an external sample of pulmonary experts across sSA in the Pan-African Thoracic Society (PATS) (figure 1).
Flow diagram of study design and progression of questionnaire development (final questionnaire provided in supplementary material). PATS: Pan-African Thoracic Society.
Literature review
A literature review was performed and publicly available ILD questionnaires reviewed in October, 2019. Publicly available ILD questionnaires were identified through a PubMed and Google search and revealed two publicly available ILD questionnaires, both developed in high-income countries [12, 13]. Our search did not identify any exposure questionnaires related to ILD within sSA. Next, a review of www.pneumotox.com, a comprehensive database of medications causing lung injury, was performed and medications with high incidence (>200 cases) of injury were recorded. Questionnaires and Pneumotox data were compiled and redundant items were removed; all remaining items from the two questionnaires and the Pneumotox data were eligible for inclusion and used to create an initial comprehensive list of 82 questionnaire items. These items were separated into four categories: “Smoking and Drugs”, “Environmental Exposures”, “Occupations” and “Medications”.
Face validation and pilot group ranking
The 82-item list was face-validated and modified by three pulmonary experts in Uganda in November, 2019. This panel included the principal investigator (PJ) and two senior pulmonary physicians at Makerere Lung Institute in Kampala, Uganda. These physicians reviewed the list for broad applicability to sSA and ensured that the questionnaire structure was applicable for clinic use in sSA.
Next, the questionnaire was distributed to a pilot group, a convenience sample of 10 physicians familiar with ILD at Makerere School of Medicine and the associated tertiary medical centre, Mulago Hospital, in Kampala, Uganda. The group ranked the question items using a five-point Likert scale (5 indicating a highly important and locally relevant item, and 1 indicating an item that was not locally relevant).
The pilot group was instructed to focus on the likelihood that the exposure item contributes to ILD and was relevant to their local setting. The group was also encouraged to suggest any items they felt were missing in each category.
Focus group discussion
A focus group discussion guide was developed by the study team (PJ, NAR and RP) to explore topics such as item understanding, response/recall difficulty, contextual relevance, potential stigmatisation, validity of suggested items, questionnaire structure, and barriers and facilitators to the feasibility and acceptability of questionnaire use in local clinical practice. Focus group discussion participants were recruited via posted fliers and snowball sampling among medical personnel at Makerere University, Kampala, Uganda. In addition, to allow for iterative discussion per the qualitative Delphi protocol, several members of the initial pilot group were included in the focus group discussion [11]. The focus group discussion was conducted with 12 Ugandan medical providers and a note-taker (RP) and moderator (PJ) in Uganda (table 1).
Group demographics
All questionnaire items were reviewed, including items newly suggested by the pilot group. Questionnaire items ranked in the lower quartile of importance or with high variability (defined as having a standard deviation in the upper quartile and indicating disagreement within the pilot group on level of importance) were addressed specifically by the moderator (PJ). The focus group discussion was audio recorded and transcribed in a condensed form by two members of the study team (PJ and RP). The transcription process focused on discussions related to inclusion or exclusion of questionnaire items, feasibility and structure. Emerging themes related to consensus or disagreement about questionnaire items were analysed using a grounded theory approach that describes a systematic inductive method for analysing qualitative data and developing codes to identify recurrent themes [14].
Two members of the study team (PJ and RP) conducted focus group discussion analysis. As themes were identified and categorised, a codebook was iteratively developed. The two reviewers compared codes; discrepancies were discussed to reach consensus. Finally, the coded transcript was reviewed along with original audio by another member of the study team (TS) to ensure no disagreements existed. Based upon this qualitative analysis, items with consensus for omission or addition as well as structural changes to the questionnaire were implemented.
External validation
Questionnaires derived from the steps above were distributed to the PATS in July 2020. The PATS is a multidisciplinary group of pulmonary specialists from across sSA. The questionnaire was modified for electronic administration and distributed to all PATS members via e-mail. Members individually ranked each item on a five-point Likert scale. The data were analysed and questions with a mean Likert scale ranking in the lowest decile were removed. Three duplicate items were added to allow for increased strength to calculate Cronbach's α in three item groups (asbestos: one addition for a total of four items; water damage: one addition for a total of three items; chemical fumes: one addition for a total of three items). Additionally, questions with high discordance were all reviewed by the study team. Following this analysis, the final questionnaire was produced.
Ethical approval for this project was provided by Johns Hopkins University (IRB #00225975) and the Makerere School of Medicine Research and Ethics Committee (#2019-157) in November 2019.
Results
Face validation
The initial questionnaire included 82 unique questions in four categories: “Smoking and Drugs”, “Environmental Exposures”, “Occupations” and “Medications”. Face validation among three pulmonary physicians practicing in Uganda resulted in five item additions and five deletions. The reason for item removal was lack of relevance to the local context in all five cases. The five additions were based upon clinical experience of the three preliminary reviewers.
Pilot group Likert ranking
The pilot group included 10 Ugandan physicians from four specialties who were all involved with ILD treatment or diagnosis (table 1). The mean±sd item Likert scale score was 4.08±0.51, median 4.11 (interquartile range (IQR) 3.79–4.42). 19 questions ranked in the lowest quartile for importance and relevance; 20 questions had high variability or disagreement (upper quartile of standard deviation >1.24). After accounting for overlap, 27 items were flagged for discussion within the focus group discussion. The pilot group provided 30 new and unique additions. Redundant suggestions were combined after review by the study team, e.g. recommendations for “dusty road”, “road dust” and “dirt road” were combined as the phrase “dusty road”.
Focus group discussion
The focus group discussion was conducted in February 2020 in Uganda with 12 participants (table 1). Coding and analysis revealed consensus for 22 re-wording or restructuring modifications, 14 additions and 10 deletions. When discussing the structure of the questionnaire, the primary theme was the desire for an exhaustive questionnaire despite feasibility concerns (table 2).
Examples of primary themes identified during focus group discussion
Modifications: There were four primary themes for questionnaire modifications: poor local relevance (most common), lack of comprehension or familiarity for patients, lack of comprehension or familiarity for healthcare providers and stigma sensitivity. For example, when discussing inclusion of “parakeet”, one focus group discussion participant stated “but why be specific and have these birds we do not know, start with birds you know: chickens, ducks, turkeys, pigeons, guinea fowl.” This modification reached unanimous consensus. Modifications included restructuring question 2 (smoking questions) and question 8 (indoor smoke exposure) as single questions with sub-headings; this resulted in seven previously independent items being combined within these two questions (supplementary questionnaire).
Additions: The Occupations and Environmental Exposures categories had the most suggested additions, five of 14 and four of 14, respectively, with only two additions in the Medications category and one in the Smoking and Drugs category. Examples of suggested additions included “sniffing petrol”, “shisha”, “chalk dust” and “charcoal seller”.
Deletions: Deletions were most common in the Environmental Exposures category (five of 10, 50%). The most common reason for deletion was lack of local relevance; some examples of deletions include “humidifier”, “swamp cooler” and “beryllium” (supplement 1).
Focus group discussion resulted in a questionnaire with 81 unique items. Three duplicate items were added prior to PATS ranking to allow for calculation of internal validity; this resulted in 84 items for PATS Likert ranking (see below).
PATS Likert ranking
In the final step, 58 members of PATS (table 1) completed an electronic Likert ranking of the 84 items. Most respondents (85%) practiced primarily in urban settings; respondents represented 17 countries (figure 2). Most respondents were pulmonary physicians; however, five specialities and four occupations were represented (table 1). The mean±sd item Likert ranking was 3.91±0.37, median 4.02 (IQR 3.65–4.17). The mean Likert ranking of the 14 additions from the focus group discussion and pilot group were above average (3.93±0.82). Eight questions in the lowest decile for importance (<3.35) were removed from the final questionnaire (supplement 1). The item category with the lowest mean ranking was Environmental Exposures; this was consistent across all phases of questionnaire development. Duplicate item groups analysed for internal validity by calculation of Cronbach's α were asbestos (four items, Cronbach's α=0.893), water damage (three items, Cronbach's α=0.851) and chemical fumes (three items, Cronbach's α=0.922). Suggestions for additions elicited 40 unique items; no item had more than three PATS participants suggest it. Additionally, no suggested item had been omitted in prior phases of development. Ultimately, no PATS additions were included in the final questionnaire (figure 1). After removal of the lowest decile and duplicate questions, the final questionnaire consisted of 73 items.
Location of respondents. Marker size proportional to number of respondents.
Discussion
We describe the development of a locally relevant ILD questionnaire for sSA through a mixed-method qualitative Delphi approach. To date, there are limited data on exposure instruments for ILD in low-middle-income country settings. Several population-based studies focused on preserved ratio impaired spirometry have included questionnaires on biomass air pollution and history of tuberculosis [15–17]. While many of these studies have assessed occupational and environmental exposures and chronic obstructive pulmonary disease, few addressed risk factors for restriction or ILD. In the ILD-India Registry, Singh et al. [18] used a questionnaire and demonstrated 47% of participants had probable hypersensitivity pneumonitis, with 48% reporting exposure to air coolers, 26% to air conditioners, 21% to birds and 20% to moulds at home.
Development of exposure questionnaires is challenged by external validation because objective measurement of historical exposures is generally not possible. Generally, exposure questionnaires are more reliable if real-world exposure data or objective markers are correlated to questionnaire responses [19]. For example, in a study of exposure to solvents, Tielemans et al. [8] used urinary metabolites of solvents to confirm self-report. While objective measure of exposures is valuable, there are limitations to this approach for ILD. First, many exposures linked to ILD are difficult to measure and do not have metabolites [20]. Second, exposures linked to some types of ILD occur over years and inflammation may persist after exposure to the inciting agent has ceased [20, 21].
This study had several strengths including a large sample size of respondents across sSA. We employed a mixed-method design to allow for group interaction when forming consensus, which has some benefits beyond anonymous ranking in a classic Delphi protocol. The use of focus group discussion allows for cross-cultural communication, which can be beneficial when developing questionnaires to discuss question phrasing, structure and applicability [11, 22, 23].
To mitigate the possible bias introduced by the initial questionnaire construction, we employed a systematic method for item inclusion and obtained face validation early with local experts. In addition, subjects in our pilot group were familiar with ILD and practiced at a large tertiary centre in Kampala, Uganda; we purposely included some of these members along with randomly selected medical workers with less expertise in ILD within the focus group discussion. This allowed for iterative discussion of pilot group ranking while enhancing applicability of the questionnaire for practitioners in various settings.
To measure internal reliability, the attentiveness of respondents and their understanding, we used repetitive question items in our PATS group. Three repetitive question groups showed consistent responses, with Cronbach's α >0.80 in all question groups. This suggests participants were attentive while ranking items and understood repetitive items as similar. While further data are needed during patient administration to ensure reliability across patient care settings, high internal validity demonstrates that participants were consistent in their rankings of specific exposures and allays concerns related to question fatigue or cross-cultural understanding of questionnaire items.
Ultimately with any Delphi or other consensus building methodology, the questionnaire represents only expert opinion and can be overturned should additional objective data come to light [24]. While including only Ugandan practitioners in the early phases of questionnaire development might limit external validity across sSA, the additions by our pilot group and focus group discussion were highly ranked by PATS participants. Additionally, no item was suggested by more than three PATS participants and no omitted item was subsequently proposed for inclusion by PATS members. These data suggest applicability across sSA; however, we do not assert that this questionnaire is an all-inclusive tool or that this alone is sufficient for ILD diagnosis, which requires clinical evaluation, radiology and laboratory data in addition to exposure screening. This questionnaire is best used as a template to assist clinicians in sSA when evaluating ILD patients. It is possible that practitioners within some regions of sSA may need to tailor the questionnaire to their clinical experience.
In this study we used a mixed-method qualitative Delphi method to create an ILD exposure questionnaire in sSA. We included experts from across sSA and developed a 73-item instrument with four categories: Smoking and Drugs, Environmental Exposures, Occupations and Medications. While this questionnaire will require implementation in research participants and clinic patients with analysis to ensure reliability, it represents a useful tool for clinical care and research of ILD in sSA.
Supplementary material
Supplementary Material
Please note: supplementary material is not edited by the Editorial Office, and is uploaded as it has been supplied by the author.
Supplementary material 00205-2022.SUPPLEMENT
Questionnaire 00205-2022.QUESTIONNAIRE
Acknowledgement
We would like to thank all members of the Pan-African Thoracic Society ILD Exposure Questionnaire Working Group.
Footnotes
Provenance: Submitted article, peer reviewed.
Data availability: De-identified, individual-participant data that underlie the results reported in this article will be made available beginning 3 months and ending 5 years after publication to researchers who provide a sound proposal for use of these data. Inquiries and requests can be made to the corresponding author, P. Jackson, at peter.jackson{at}vcuhealth.org. Data use agreements may be necessary depending on the nature of the proposed use.
Ethical approval for this project was provided by Johns Hopkins University Institutional Review Board (number 00225975) and the Makerere School of Medicine Research and Ethics Committee (number 2019-157) in Kampala, Uganda, in November 2019. All participants gave informed consent prior to participation in each phase of this study.
Author contributions: All authors contributed to this work in accordance with International Committee of Medical Journal Editors guidelines. This included assisting with study design, questionnaire development, navigation of ethical review boards and protocols, data review and analysis, and contributions to this manuscript. All authors approved of the final manuscript and agree to be accountable for all aspects of the work.
Conflict of interest: P. Jackson has nothing to disclose.
Conflict of interest: R. Padalkar has nothing to disclose.
Conflict of interest: W. Katagira has nothing to disclose.
Conflict of interest: K. Mortimer has nothing to disclose.
Conflict of interest: N.A. Rykiel has nothing to disclose.
Conflict of interest: N.M. Robertson has nothing to disclose.
Conflict of interest: S.L. Pollard has nothing to disclose.
Conflict of interest: P. Alupo has nothing to disclose.
Conflict of interest: W. Checkley has nothing to disclose.
Conflict of interest: B. Kirenga has nothing to disclose.
Conflict of interest: T. Siddharthan has nothing to disclose.
Support statement: P. Jackson was supported by the Fogarty International Center and National Heart, Lung, and Blood Institute, and National Institute of Neurological Disorders and Stroke and of the National Institutes of Health under Award Number D43 TW009340. Funding information for this article has been deposited with the Crossref Funder Registry.
- Received April 28, 2022.
- Accepted June 6, 2022.
- Copyright ©The authors 2022
This version is distributed under the terms of the Creative Commons Attribution Non-Commercial Licence 4.0. For commercial reproduction rights and permissions contact permissions{at}ersnet.org
References
