International use of objective structured clinical examinations in respiratory training: a European Respiratory Society early career member survey

To the Editor:

Objective structured clinical examinations (OSCEs) are profoundly changing the way medical students are trained and assessed worldwide. Early career members (ECMs) are at the forefront across countries in their implementation. In addition to core knowledge, medical students need to acquire competencies such as good clinical reasoning, communication and interpersonal behaviour [1]. These skills are poorly assessed by written examination [2]. Therefore, OSCEs have been developed specifically to assess these competencies [3, 4]. With OSCEs, medical trainees are evaluated during simulated realistic clinical situations [3, 4]. OSCEs for a long time have been the gold-standard method to assess competencies in the USA and Canada [5, 6]. Worldwide, the use of OSCEs is expanding at each level of medical education (pre-, post-graduation and continuous medical education). However, differences in OSCE delivery may exist within and across different countries, leading to disparities in medical training organisation. The use of OSCEs in respiratory medicine is particularly relevant given the spectrum of diseases and clinical situations that respiratory physicians must face during their daily practice and throughout their career. Moreover, several technical skills need to be acquired such as bronchoscopy or pulmonary function tests [7, 8]. We have recently reported the results of a French national survey among respiratory teachers on the use of OSCEs for students’ training and assessment. It confirmed a wide heterogeneity in the use of OSCEs as only 50% used OSCEs for medical education [9]. Data regarding the use of OSCEs in respiratory medicine around the world are sparse. As the European Respiratory Society (ERS) aims to improve respiratory medical education, we sought its support to conduct a survey regarding current international practice of OSCEs. Our aim is therefore to describe how OSCEs are used in respiratory medicine training across Europe and around the world to identify potential levers of improvement.

After analysis and validation of the project by the ERS Early Career Member Committee (ECMC) and with its support, we conducted an anonymous online survey. ECMs are ERS members under 40 years of age, either students or professionals involved in respiratory diseases, both medical and paramedical staff, from Europe or all over the world. The survey was generated using the Microsoft Form® tool. The survey included 29 questions divided into three chapters: 1) general information on respondents (country, present position, involvement in teaching); 2) general knowledge of respondents regarding OSCEs (awareness, experience, benefits for the career); and 3) use of OSCEs in respondents’ institutions (use for training and/or evaluation, for how long). The survey was disseminated through the ERS ECMC mailing list (9724 ECMs registered on the mailing list) and was sent twice. Additional communication was made through each ERS assembly using mailing lists and social media. Only ECMs were allowed to answer. Responses were collected from 1 December 2021 to 28 February 2022.

Overall, 146 physicians from 42 different countries around the world responded to the survey (figure 1). The number of respondents per country varied greatly, from 1 to 29 (mean±sd 4.0±5.2). The majority of them originated from Europe (n=123 (84%), from 25 different countries, see figure 1). Their mean±sd age was 34.8±3.8 years. The majority of respondents were adult pulmonologists (n=108, 74%). Other participating physicians were: allergologists (6, 4%), paediatricians (16, 11%), thoracic surgeons (3, 2%), cardiologist (1, 0.7%), oncologist (1, 0.7%) and physiologist (1, 0.7%). Paramedical respondents were physiotherapists (6, 4%), nurses (2, 1%) and one researcher. 110 (75%) participants were medical doctors, 25 (17%) PhD students, 6 (4%) residents and 5 (3%) were associated professors. 97 (66%) respondents were involved in medical teaching in their local institution, but only 40 out of 97 (41%) had received specific training in medical teaching. The majority of the physician responders (99, 68%) had undergone a competency evaluation during their medical studies, and 53 (54%) passed an OSCE.

• Download figure
• Open in new tab
• Download powerpoint

FIGURE 1

a) World and b) European maps of objective structured clinical examination (OSCE) practice among European Respiratory Society early career member respondents. Orange colour: countries in which OSCEs are practised in at least one university or hospital of respondents to the survey; yellow colour: countries in which OSCEs are not practised in any university or hospital of respondents to the survey; grey colour: countries with no respondents to the survey. c) Subjective evaluation of the OSCEs among respondents personally involved in the OSCEs (n=39).

At the time of the survey, 81 (55%) respondents were aware of the use and value of OSCEs. OSCEs were used in at least one institution in 30 out of 42 countries. For 67 (46%) respondents, OSCEs were already implemented in their institution (hospital or university) at the time of the survey, but less than half of these ECMs had received a specific training in OSCEs (31 out of 67 (46%)). The year of implementation of OSCEs in these various institutions varied between 1975 and 2021, with a mean existence of 6±8 years. We noticed a recent increase in the number of countries that have implemented OSCEs. Respondents reported the use of OSCEs for medical evaluations (62 out of 67 (92%)) and/or for training (51/67 (76%)). Respondents’ participation in OSCEs included: acting as a patient (n=13), being an examiner (n=33), or creating one or more stations (n=21, mean 3.3 stations). The main reported pitfall of the implementation of OSCEs was its time-consuming character, although it was seen as a very highly objective tool to evaluate competencies (figure 1). Thus, OSCEs were highly recommended by responders for students’ training and evaluation (figure 1).

Thanks to the ERS ECMC this is the first study to give an international view on the practice of the assessment of medical students’ competencies using the OSCE method. Almost half of the respondents use this method, and it was used in most of the countries represented in this survey. Among those who use it, OSCE was seen as a highly valued method for their own training and for training future physicians. Moreover, the use of OSCEs has increased over the last 20 years. In addition, this survey highlights the lack of training in this innovative modality for medical education received by teachers in respiratory diseases. Indeed, <50% of responders involved in medical teaching were trained in medical education, and among those working in institutions where OSCEs are practised, only 46% had received training in this mode of assessment. The main limitation of this work is the low number of responses and the incompleteness of data in figure 1, as not all countries or institutions were represented. Indeed, the ERS, although international, is above all European, which explains why European ECMs are the main respondents to this survey. On the other hand, a response bias might exist, as only ECMs were interviewed and among them the most interested in the subject responded more readily to the survey than others. However, the impression that this educational tool is widely used at least in Europe seems correct, and we can assume that asking ECMs is a way to address people who are aware of the implementation of OSCEs in their institution and their country. Among OSCE users we could have observed different answers on the pedagogical interest of this tool if we had not restricted the survey to the ECMs of the ERS. In addition to medical knowledge, it is essential to assess the physicians’ skills, particularly in the field of respiratory diseases, which involves many technical procedures. In order to homogenise the use of OSCEs throughout Europe, where it is highly appreciated by the ECMs, a task force of standardisation of their practice could be initiated, through learned societies such as the ERS, for example. A common session about OSCEs during the ERS congress could also be set up to complete the assessment on the implementation of OSCEs among ECMs in different countries and propose common training. A multinational and multi-institutional pooling of OSCE scenarios could be performed, with the aim of standardising, validating and facilitating their use among the community of respiratory physicians. Setting up the first international educational respiratory tool would help to cement our community.