Development and initial validation of a disease-specific instrument to measure health-related quality of life in hypersensitivity pneumonitis

PROM development pathway

The development of the HP-HRQOL instrument followed the steps outlined in the Food and Drug Administration (FDA) guidelines for development of a new PROM in a target population [12] (figure 1). Briefly, in our prior work, we developed a conceptual framework of living with HP using data from 18 qualitative interviews where we identified six major themes representing impacts on HRQOL [11]. Guided by the major themes, we created an initial set of 39 items that represented each theme. We performed cognitive debriefing interviews where we revised the items, refined the overall instrument and evaluated readability statistics with patient input [13]. The current study describes the next steps in the pathway of PROM development by assessing the validity and reliability statistics of the HP-HRQOL in a cohort of patients with HP.

• Download figure
• Open in new tab
• Download powerpoint

FIGURE 1

Study flow diagram. HP-HRQOL: hypersensitivity pneumonitis health-related quality of life.

Study design, setting and participants

This was an observational study that recruited participants living with HP who were ≥18 years of age at Weill Cornell Medicine (WCM) in New York City, NY, and The University of Virginia in Charlottesville, VA, between July 2020 and February 2023. All patients with a confirmed diagnosis of HP were eligible for screening. HP was the primary pulmonary diagnosis, and the diagnosis of HP was made via expert multidisciplinary consensus based on the combination of clinical history, high-resolution computed tomography scan, lung function and pathology where applicable. Patients were required to be English speaking and were excluded if they were unable to complete questionnaires due to cognitive impairment as determined by their treating clinician. All data collection and study visits were conducted by the WCM site, using remote methods (e.g., telephone contact and electronic completion of surveys) most frequently. All sites obtained institutional review board approval (WCM IRB# 1905020233, UVA IRB HSR# 23633).

Measures and data collection

The 39-item HP-HRQOL was administered at the first study visit and 2 weeks later. Two additional PROMs were administered at enrolment to be used as anchors. The King's Brief ILD (KBILD) questionnaire has been developed and validated in ILD [14]. It consists of 15 items across three domains: breathlessness and activity, chest symptoms and psychological. It has been administered in both paper and electronic format [15]. The 12-item Short-Form Health Survey (SF-12) is a self-report instrument that assesses eight domains of health and wellbeing that comprise two different summary scores: the physical component score (PCS) and mental component score (MCS). The SF-12 has been validated for use in both paper and electronic format [16]. Enrolled participants personally completed questionnaires (electronically or on paper) on two separate occasions, ∼2 weeks apart. Additional medical data (including PFTs, 6-min walk distance, computer tomography scans, medications) and demographic data were abstracted from patient's medical charts by a member of the research team.

Response scale and scoring

A 5-point Likert scale was used for all response options throughout the development of the HP-HRQOL (online supplementary material B). Each item was scored out of 5 points and, for items surviving item reduction steps, summation scoring was used to derive domain scores. The overall score was calculated by adding individual domain scores for the items that were part of the final factor solution. Lower scores indicated worse quality of life and higher scores indicated better quality of life. The means for each domain and the total score were reported separately.

Structural validity: domain refinement and item reduction

To minimise respondent burden, we evaluated the initial list of 39 items for redundancy using a correlation matrix. For items with a correlation >0.8, the weaker item (as determined by prior content and face validity testing) was eliminated [17]. We evaluated the remaining items individually for floor (minimum) and ceiling (maximum) effects using a conservative threshold of 25% [18]. Items with “1” or “5” responses above the pre-specified threshold were considered for elimination, and exceptions were discussed at length among the research team and a final determination was made based on clinical judgement. Surviving items were subjected to an exploratory factor analysis (EFA) to determine the number of dimensions underlying the item set and how each item related to the dimensions. We used a principal axis method and orthogonal varimax rotation to extract factors that were uncorrelated. Orthogonal rotation was used to maximise the association of each item with a single factor [19]. Individual items with dominant factor loadings >0.6 on a single factor were retained and summed to create the total factor (domain) score. Any exceptions were made in discussion with the research team and based on clinical judgement [20].

Reliability

Internal consistency reliability was assessed on the individual domains using Cronbach's coefficient α [21]. We considered α>0.7 as representing a high degree of internal consistency. Test–retest reliability (TRT) was calculated using intraclass correlation coefficients (ICC) on scores from participants who completed the baseline and 2-week follow-up survey. Participants who were documented to have a respiratory illness, hospitalisation or change in respiratory medication between the time of the first and second survey administration were excluded from the TRT. A value >0.7 was used as the accepted cut-off for TRT [22].

Validity

Concurrent validity (how well the scores of the HP-HRQOL compare with a measure that has validity data available in HP) was assessed using a Spearman correlation coefficient between the HP-HRQOL scores and several anchors collected at baseline. Physiological anchors included % predicted forced vital capacity (FVC) and % predicted diffusing capacity of the lungs for carbon monoxide (D_LCO). Owing to restrictions in performing lung function tests as a part of clinical research during the early COVID-19 pandemic, the FVC and D_LCO values were collected from most recent values documented as a part of routine clinical care. PROM anchors included: the SF-12 [23, 24] and the KBILD instruments [14]. We evaluated each domain of the HP-HRQOL and the sum score with each domain and total score of the KBILD and SF-12. We considered r<0.4 to indicate a weak correlation, r≥0.4<0.7 to indicate moderate correlation and r>0.7 to indicate a strong correlation [25, 26].

Known groups validity was assessed using analysis of variance (ANOVA) in which we compared the mean HP-HRQOL score as a dependent variable across categories of several variables: 1) known/unknown antigen; 2) use of supplemental oxygen; 3) % predicted FVC (<50%, 50–80%, >80%); 4) % predicted D_LCO (moderate to severe ≤59%, mild and normal ≥60%); 5) use of systemic steroids, immunosuppressants, neither, or both; and 6) use of antifibrotic drugs. We hypothesised that participants with worse lung function and those using supplemental oxygen would have worse HP-HRQOL scores than those with better lung function and not using supplemental oxygen. We hypothesised that those taking immunosuppressants and antifibrotics would have higher HP-HRQOL scores than those not taking those medications. We hypothesised that there would be no difference in scores between patients with known and those with unknown antigen.