Skip to content

400+ Validation Studies

Evidence-based Simulation Training

Find here a selection of validation studies, the culmination of extensive research and rigorous validation processes providing evidence of the validity and reliability of our simulation technology, which helped pave the way for revolutionizing surgical education and training. Based on some of the studies we have established proficiency-based curricula which are integrated into our simulators.



Filter by Simulator

Simulator Training for Endobronchial Ultrasound: A Randomized Controlled Trial Konge L. et al., Eur Respir J. 2015

Endobronchial ultrasound-guided transbronchial needle aspiration (EBUS-TBNA) is very operator dependent and has a long learning curve. Simulation-based training might shorten the learning curve, and an assessment tool with solid validity evidence could ensure basic competency before unsupervised performance.

A total of 16 respiratory physicians, without EBUS experience, were randomised to either virtual-reality simulator training or traditional apprenticeship training on patients, and then each physician performed EBUS-TBNA procedures on three patients. Three blinded, independent assessor assessed the video recordings of the procedures using a newly developed EBUS assessment tool (EBUSAT).

The internal consistency was high (Cronbach’s α=0.95); the generalisability coefficient was good (0.86), and the tool had discriminatory ability (p<0.001). Procedures performed by simulator-trained novices were rated higher than procedures performed by apprenticeship-trained novices: mean±sd are 24.2±7.9 points and 20.2±9.4 points, respectively; p=0.006. A pass/fail standard of 28.9 points was established using the contrasting groups method, resulting in 16 (67%) and 20 (83%) procedures performed by simulator-trained novices and apprenticeship-trained novices failing the test, respectively; p<0.001.

The endobronchial ultrasound assessment tool could be used to provide reliable and valid assessment of competence in EBUS-TBNA, and act as an aid in certification. Virtual-reality simulator training was shown to be more effective than traditional apprenticeship training.


Construct Validity of the BRONCH Mentor Simulator for Essential Bronchoscopic Skills Pastis NJ et al., J Bronchology Interv Pulmonol. 2014

Background: Although simulation-based bronchoscopy has been shown to be an effective training modality, formal assessment should still be performed as new technology emerges. We sought to validate a simulator in essential bronchoscopic tasks, and survey perceptions of bronchoscopists on simulation.

Methods: A cohort study at 2 medical centers used 3 groups to assess construct validity of the Simbionix Bronchoscopy Simulator: 7 first-year fellows with <10 bronchoscopies each (novice), 6 pulmonologists with 200 to 1000 bronchoscopies each (experienced), and 7 pulmonologists with >1000 bronchoscopies each (expert). Participants were tested in 4 tasks (1: scope manipulation, 2: guided anatomic navigation, 3: airway anatomy, and 4: lymph node anatomy). Participants were scored and surveyed on their impressions of simulation. The means and Kruskal-Wallis test among groups were compared by task item (P<0.05).

Results: There were statistically significant differences in mean ranks among groups for tasks 1 and 3. For task 1, final score, time, mid-lumen time, and wall hits were discriminative (P=0.006, 0.006, 0.012, and 0.014, respectively). For task 3, time, bronchial segments identified, bronchial segments incorrectly identified, and bronchial segments skipped were discriminative (P=0.04, 0.012, 0.013, and 0.013, respectively). There was no statistically significant difference for task 2 and task 4. All participants agreed that simulation training is helpful and should be incorporated into bronchoscopic training.

Conclusions: The simulator demonstrated validity in differentiating skill in scope manipulation and airway anatomy, but did not discriminate skill levels in anatomic orientation or identification of lymph nodes. Bronchoscopy simulation was viewed as helpful by all levels and should be considered before performance on patients.


Simulation-Based Bronchoscopy Training Kennedy CC, Maldonado F., Cook DA, CHEST July 2013, P183-192.

Background: Simulation-based bronchoscopy training is increasingly used, but effectiveness remains uncertain. We sought to perform a comprehensive synthesis of published work on simulation-based bronchoscopy training.

Methods: We searched MEDLINE, EMBASE, CINAHL, PsycINFO, ERIC, Web of Science, and Scopus for eligible articles through May 11, 2011. We included all original studies involving health professionals that evaluated, in comparison with no intervention or an alternative instructional approach, simulation-based training for flexible or rigid bronchoscopy. Study selection and data abstraction were performed independently and in duplicate. We pooled results using random effects meta-analysis.

Results: From an initial pool of 10,903 articles, we identified 17 studies evaluating simulation-based bronchoscopy training. In comparison with no intervention, simulation training was associated with large benefits on skills and behaviors (pooled effect size, 1.21 [95% CI, 0.82-1.60]; n=8 studies) and moderate benefits on time (0.62 [95% CI, 0.12-1.13]; n=7). In comparison with clinical instruction, behaviors with real patients showed nonsignificant effects favoring simulation for time (0.61 [95% CI, -1.47 to 2.69]) and process (0.33 [95% CI, -1.46 to 2.11]) outcomes (n=2 studies each), although variation in training time might account for these differences. Four studies compared alternate simulation-based training approaches. Inductive analysis to inform instructional design suggested that longer or more structured training is more effective, authentic clinical context adds value, and animal models and plastic part-task models may be superior to more costly virtual-reality simulators.

Conclusions: Simulation-based bronchoscopy training is effective in comparison with no intervention. Comparative effectiveness studies are few.


Standardization of Bronchoscopy Training Across Yorkshire and Humber Deanery F Chowdhury, S Pathmanathan, I Aslam, A Arnold, J Hogg, S Renshaw, J Hill, N Chaudhuri, T Rogers, J Kastelik. The abstract was presented at the British Thoracic Society June 13-14, 2013, University Place Conference Centre, University of Manchester, UK

Background Reduced exposure to bronchoscopyis a key issue for respiratory trainees with effect on their confidence in undertaking the procedure and thus patient safety. Studies have shown that simulation improves confidence in bronchoscopy skills but do not explore the most optimal teaching methods.

Aim To assess two different methods of delivering bronchoscopy simulation training

Methods Two half day simulation bronchoscopy courses were designed independently within the Yorkshire and Humber Deanery. Course 1 concentrated on providing a knowledge based training consisting of a didactic lecture followed by equal time spent on a Symbionixsimulator and on the BTS e-learning hub website. Course 2 provided pre-course material in the form of BTS guidelines and bronchoscopy procedure pocketbook. The course focused on hands-on simulation training using a bronchoscopy manikin and the Symbionix simulator. All candidates completed pre and post course Likert scale questionnaires in six areas relating to participant knowledge and confidence in using a bronchoscope.

Results Overall 30 trainees; 15 in each course were evaluated. Candidates had performed between 0 to >300 previous bronchoscopies and were from across the SpR years. Both courses delivered significant improvement in confidence scores in all of the six areas assessed. The greatest improvement was found in confidence levels in technical ability . Course 1 candidates showed a greater confidence improvement in factual skills (such as knowledge of contra-indications of the procedure and anatomy). Course 2 demonstrated that 93% of candidates agreed that the simulator helped to improve technical ability in contrast to 100% with manikin exposure. 100% of candidates found the pocketbook was a useful adjuvant to the course with 93% agreeing that they would find this useful to complement their training.


Skip to navigation