Skip to content

400+ Validation Studies

Evidence-based Simulation Training

Find here a selection of validation studies, the culmination of extensive research and rigorous validation processes providing evidence of the validity and reliability of our simulation technology, which helped pave the way for revolutionizing surgical education and training. Based on some of the studies we have established proficiency-based curricula which are integrated into our simulators.



Filter by Simulator

Training simulator efficacy in developing thoracic and general surgical skills in a residency programme: a pilot study Sara Grossi et al., Eur J Cardiothorac Surg .Feb 2024

Objectives: Virtual training simulators have been introduced in several surgical disciplines to improve residents’ abilities. Through the use of the LapSim® virtual training simulator (Surgical Science, Göteborg, Sweden), this study aims to plan an effective learning path in minimally invasive thoracic and general surgery.

Methods: All thoracic and general surgery trainees in their 1st and 2nd year of residency at the University of Insubria were enrolled and randomized into 2 groups: residents undergoing an intensive twice-a-week virtual training programme (group A: n = 8) and those undergoing a once-weekly non-intensive virtual training programme (group B: n = 9). The virtual training programme was divided into 4 modules, each of 12 weeks. In the 1st module, trainees repeated grasping, cutting, clip application, lifting and grasping, and fine dissection exercises during each training session. Seal-and-cut exercise was performed as the initial and final test. Data on surgical manoeuvres (time and on mistakes) were collected; intra- and inter-group comparisons were planned.

Results: No significant differences were observed between groups A and B at the 1st session, confirming that the 2 groups had similar skills at the beginning. After 12 weeks, both groups showed improvements, but comparing data between initial and final test, only Group A registered a significant reduction in total time (P-value = 0.0015), left (P-value = 0.0017) and right (P-value = 0.0186) instrument path lengths, and in left (P-value = 0.0010) and right (P-value = 0.0073) instrument angular path lengths, demonstrating that group A acquired greater precision in surgical manoeuvres.

Conclusions: Virtual simulator training programme performed at least twice a week was effective for implementing basic surgical skills required for the trainee’s professional growth. Additional virtual training modules focused on more complex exercises are planned to confirm these preliminary results.


Differences between gynecologists and medical residents in basic motor skills of an exercise module of a virtual simulation system Freyre-Santiago JM. Ginecol Obstet Mex, July 2022
OBJECTIVE: To evaluate the differences in the outcomes of the exercises of the basic motor skills module of the Simbionix LAP Mentor simulator between a group of gyne- cology residents and another group of already graduated specialists.
MATERIALS AND METHODS: Observational, prospective, longitudinal and compara- tive study carried out from December 2019 to January 2020 at the Hospital Español de México. Gynecology residents and specialists in the same branch with diverse experience in laparoscopic surgery were selected. The exercises of the basic motor skills module of a virtual reality system were evaluated with a high-fidelity simulation training program. The Student’s t test was applied for variables with normal distribution and the Mann-Whitney U test for variables that did not meet the normality criterion.
RESULTS: Thirty-one participants were included: the specialist group (n = 19) and the resident group (n = 12). Significant differences between residents and specialists were found in exercise 3-eye-hand coordination (4.45 sec, 95%CI: 0.167-8.73; p < 0.05) and in exercise 5-pulling and stapling of leaking hoses (29.58 sec, 95%CI: -42.99 -14.00; p < 0.001), in favor of the specialist group. In exercise 7-cutting and exercise 8-fulguration there were no significant differences between one group and the other.
CONCLUSIONS: The Simbionix LAP Mentor virtual reality simulator detects differences between groups of obstetrician-gynecologists with different experience in minimally invasive gynecologic surgery and residents of the specialty.


Boosting Confidence: Enhancing Spinal Cord Stimulator Needle and Lead Placement Through Simulation Training Ali S et al.. Cuerus, March 2024


This pilot study aims to examine the effectiveness of a spinal cord stimulator (SCS) simulator training system in improving the confidence of pain fellows in SCS placement.


Five Ukrainian physicians (neurologists, neurosurgeons, and an anesthesiologist) completed a 10-item survey regarding their confidence in various aspects of SCS placement and their opinions on how effective SCS models were for educational purposes. After placing SCS leads using the SCS simulator, the physicians took the same survey again. The Mann-Whitney U test was used to determine if there was a significant difference in total scores pre and post-simulator training. The software PAST (PAleontological STatistics) was used for statistical analysis.


Overall, five participants had a 38% statistically significant increase in survey scores before and after the intervention (mean: 4.2 vs. 6.2, p = 0.0055). With regards to each item of the survey, participants had a significantly increased confidence in driving leads (2.6 vs. 5.2, p = 0.008) and in overall technical skills for the SCS procedure after the training (2.8 vs. 5.2, p = 0.0188). Although the other eight survey items were not statistically significant (p > 0.05), participants had a 28% increase in confidence when inserting epidural needles, a 20% increase in interpreting simulated X-rays, a 32% increase in navigating challenging anatomical variations, a 12% increase in identifying key anatomical landmarks, a 20% increase in ensuring the correct placement of the lead, or a 53% increase in preparedness for performing an SCS procedure in a real clinical setting. The participant’s perspective on how valuable the stimulator training was for enhancing procedural skills increased by 38% and how well the simulator replicated real-life SCS procedure increased by 52%, although both were statistically insignificant (p > 0.05).


This pilot study shows that the utilization of simulated neuromodulation training is a viable means of augmenting neuromodulation education by increasing physician’s confidence in aspects of the SCS placement procedure. The extent to which simulator training improves procedural skills in a real-life SCS placement needs to be investigated further.


Validity of a virtual reality endoscopic retrograde cholangiopancreatography simulator: can it distinguish experts from novices? Georgiou K et al. Front Surg. 2023 Dec 6

Background: There is a lack of evidence regarding the effectiveness of virtual simulators as a means to acquire hands-on exposure to endoscopic retrograde cholangiopancreatography (ERCP). The present study aimed to assess the outcome and construct validity of virtual ERCP when training on the GI II Mentor simulator.

Methods: A group of seven experienced endoscopists were compared with 31 novices. After a short introduction, they were requested to carry out three virtual ERCP procedures: diagnosing and removing a common bile duct (CBD) stone; diagnosing and taking brush cytology from a hilar stenosis; and, finally, diagnosing and treating a cystic leakage with a BD stent. For each task, the total time required to complete the task, time required to correctly view the papilla, total time of irradiation, time to deep cannulation, time to define diagnosis, time to complete sphincterotomy, and time to complete the respective intervention were measured. Cannulation of the BD, correct diagnosis, sphincterotomy, and time to complete intervention were assessed by an assessor blinded to the status of the endoscopist who performed the virtual ERCP.

Results: The time required to visualize the papilla and to cannulate deeply when removing the BD stone was significantly shorter for the experts (both p < 0.05). The time to visualize the papilla, cannulate deeply, reach a diagnosis, complete sphincterotomy, and complete the intervention was significantly shorter for the experts when managing cystic leakage (all p < 0.05). In diagnosing and taking brush cytology from a hilar stenosis, there was only a trend toward the experts needing less time for the deep cannulation of the BD (p = 0.077).

Conclusion: The performance differed between experts and novices, especially in the management of cystic leakage. This corroborates the construct validity of the GI II Mentor simulator.


Fundamentals of endoscopic surgery: creation and validation of the hands-on test. Vassiliou MC, et al. Surg Endosc. 2013 Nov 20.

Background The Fundamentals of Endoscopic Surgery™ (FES) program consists of online materials and didactic and skills-based tests. All components were designed to measure the skills and knowledge required to perform safe flexible endoscopy. The purpose of this multicenter study was to evaluate the reliability and validity of the hands-on component of the FES examination, and to establish the pass score.

Methods Expert endoscopists identified the critical skill set required for flexible endoscopy. They were then modeled in a virtual reality simulator (GI Mentor™ II, Simbionix™ Ltd., Airport City, Israel) to create five tasks and metrics. Scores were designed to measure both speed and precision. Validity evidence was assessed by correlating performance with self-reported endoscopic experience (surgeons and gastroenterologists [GIs]). Internal consistency of each test task was assessed using Cronbach’s alpha. Test–retest reliability was determined by having the same participant perform the test a second time and comparing their scores. Passing scores were determined by a contrasting groups methodology and use of receiver operating characteristic curves.

Results A total of 160 participants (17 % GIs) performed the simulator test. Scores on the five tasks showed good internal consistency reliability and all had significant correlations with endoscopic experience. Total FES scores correlated 0.73, with participants’ level of endoscopic experience providing evidence of their validity, and their internal consistency reliability (Cronbach’s alpha) was 0.82. Test–retest reliability was assessed in 11 participants, and the intraclass correlation was 0.85. The passing score was determined and is estimated to have a sensitivity (true positive rate) of 0.81 and a 1-specificity (false positive rate) of 0.21.

Conclusions The FES hands-on skills test examines the basic procedural components required to perform safe flexible endoscopy. It meets rigorous standards of reliability and validity required for high-stakes examinations, and, together with the knowledge component, may help contribute to the definition and determination of competence in endoscopy.


Evaluation of Two Flexible Colonoscopy Simulators and Transfer of Skills into Clinical Practice Gomez PP et al.,J Surg Educ. 2014 Sep 16

Introduction: Surgical residents have learned flexible endoscopy by practicing on patients in hospital settings under the strict guidance of experienced surgeons. Simulation is often used to “pretrain” novices on endoscopic skills before real clinical practice; nonetheless, the optimal method of training remains unknown. The purpose of this study was to compare endoscopic virtual reality and physical model simulators and their respective roles in transferring skills to the clinical environment.

Methods: At the beginning of a skills development rotation, 27 surgical postgraduate year 1 residents performed a baseline screening colonoscopy on a real patient under faculty supervision. Their performances were scored using the Global Assessment of Gastrointestinal Endoscopic Skills (GAGES). Subsequently, interns completed a 3-week flexible endoscopy curriculum developed at our institution. One-third of the residents were assigned to train with the GI Mentor simulator, one-third trained with the Kyoto simulator, and one-third of the residents trained using both simulators. At the end of their rotations, each postgraduate year 1 resident performed one posttest colonoscopy on a different patient and was again scored using GAGES by an experienced faculty.

Results: A statistically significant improvement in the GAGES total score (p < 0.001) and on each of its subcomponents (p = 0.001) was observed from pretest to posttest for all groups combined. Subgroup analysis indicated that trainees in the GI Mentor or both simulators conditions showed significant improvement from pretest to posttest in terms of GAGES total score (p = 0.017 vs 0.024, respectively). This was not observed for those exclusively using the Kyoto platform (p = 0.072). Nonetheless, no single training condition was shown to be a better training modality when compared to others in terms of total GAGES score or in any of its subcomponents.

Conclusion: Colonoscopy simulator training with the GI Mentor platform exclusively or in combination with a physical model simulator improves skill performance in real colonoscopy cases when measured with the GAGES tool.

Keywords: GAGES; Patient Care; Practice-Based Learning and Improvement; Systems-Based Practice; flexible endoscopy; simulation; skills assessment.


Development of a fundamentals of endoscopic surgery proficiency-based skills curriculum for general surgery residents. Mizota T et al., Surg Endosc. Feb 2020

Background Fundamentals of Endoscopic Surgery (FES) has become a board certification requirement for general surgery residents. While the FES program provides a robust didactic curriculum, an endoscopic skills curriculum is lacking for this high stakes assessment. The aims of this study were to develop a proficiency-based endoscopic skills curriculum and assess its effectiveness on success in the FES exam.

Methods Endoscopy experts developed a multiple-choice questionnaire based on the FES online didactics. Five training cases from the GI Mentor II simulator were selected, and expert performance defined proficiency levels for each case. Participating surgery residents were required to review online didactics and achieve proficiency twice on selected simulator cases. The multiple-choice questionnaire, simulator-generated metrics of two endoscopy cases, Global Assessment of Gastrointestinal Endoscopic Skills (GAGES), NASA-Task Load Index (TLX), and the manual portion of the FES exam were used for assessment before and after training. The curriculum was implemented either alongside a clinical endoscopy rotation or independent of the rotation. Clinical endoscopic skills of participants with a dedicated rotation were assessed using GAGES.

Results Twenty-eight general surgery residents (PGY 2–5) participated in the study, of which 25 (89%) completed the curriculum. Scores of the multiple-choice questionnaire and all simulator-generated metrics improved in the post-training assessment, with the exception of the percentage of mucosal surface examined, which was slightly decreased. Simulated and clinical GAGES scores and the NASA-TLX score improved after training. Performance scores on four of five FES exam tasks were significantly improved.

Conclusions The proficiency-based endoscopic skills curriculum was successfully implemented both alongside the clinical endoscopy rotation and independent of the rotation. Participating residents acquired skills to pass the FES exam. This curriculum will be valuable to general surgery residency programs


Simulator Training for Endobronchial Ultrasound: A Randomized Controlled Trial Konge L. et al., Eur Respir J. 2015

Endobronchial ultrasound-guided transbronchial needle aspiration (EBUS-TBNA) is very operator dependent and has a long learning curve. Simulation-based training might shorten the learning curve, and an assessment tool with solid validity evidence could ensure basic competency before unsupervised performance.

A total of 16 respiratory physicians, without EBUS experience, were randomised to either virtual-reality simulator training or traditional apprenticeship training on patients, and then each physician performed EBUS-TBNA procedures on three patients. Three blinded, independent assessor assessed the video recordings of the procedures using a newly developed EBUS assessment tool (EBUSAT).

The internal consistency was high (Cronbach’s α=0.95); the generalisability coefficient was good (0.86), and the tool had discriminatory ability (p<0.001). Procedures performed by simulator-trained novices were rated higher than procedures performed by apprenticeship-trained novices: mean±sd are 24.2±7.9 points and 20.2±9.4 points, respectively; p=0.006. A pass/fail standard of 28.9 points was established using the contrasting groups method, resulting in 16 (67%) and 20 (83%) procedures performed by simulator-trained novices and apprenticeship-trained novices failing the test, respectively; p<0.001.

The endobronchial ultrasound assessment tool could be used to provide reliable and valid assessment of competence in EBUS-TBNA, and act as an aid in certification. Virtual-reality simulator training was shown to be more effective than traditional apprenticeship training.


A Multi-institution Study on the Association of Virtual Reality Skills with Continence Recovery after Robot-assisted Radical Prostatectomy Timothy NChu et al., European Urology Focus

Background: Virtual reality (VR) simulators are increasingly being used for surgical skills training. It is unclear what skills are best improved via VR, translate to live surgical skills, and influence patient outcomes.

Objective: To assess surgeons in VR and live surgery using a suturing assessment tool and evaluate the association between technical skills and a clinical outcome.

Design, setting, and participants: This prospective five-center study enrolled participants who completed VR suturing exercises and provided live surgical video. Graders provided skill assessments using the validated End-To-End Assessment of Suturing Expertise (EASE) suturing evaluation tool.

Outcome measurements and statistical analysis: A hierarchical Poisson model was used to compare skill scores among cohorts and evaluate the association of scores with clinical outcomes. Spearman’s method was used to assess correlation between VR and live skills.

Results and limitations: Ten novices, ten surgeons with intermediate expertise (median 64 cases, interquartile range [IQR] 6-80), and 26 expert surgeons (median 850 cases, IQR 375-3000) participated in this study. Intermediate and expert surgeons were significantly more likely to have ideal scores in comparison to novices for the subskills needle hold angle, wrist rotation, and wrist rotation needle withdrawal (p < 0.01). For both intermediate and expert surgeons, there was positive correlation between VR and live skills for needle hold angle (p < 0.05). For expert surgeons, there was a positive association between ideal scores for VR needle hold angle and driving smoothness subskills and 3-mo continence recovery (p < 0.05). Limitations include the size of the intermediate surgeon sample and clinical data limited to expert surgeons.

Conclusions: EASE can be used in VR to identify skills to improve for trainee surgeons. Technical skills that influence postoperative outcomes may be assessable in VR.


Construct Validity of the BRONCH Mentor Simulator for Essential Bronchoscopic Skills Pastis NJ et al., J Bronchology Interv Pulmonol. 2014

Background: Although simulation-based bronchoscopy has been shown to be an effective training modality, formal assessment should still be performed as new technology emerges. We sought to validate a simulator in essential bronchoscopic tasks, and survey perceptions of bronchoscopists on simulation.

Methods: A cohort study at 2 medical centers used 3 groups to assess construct validity of the Simbionix Bronchoscopy Simulator: 7 first-year fellows with <10 bronchoscopies each (novice), 6 pulmonologists with 200 to 1000 bronchoscopies each (experienced), and 7 pulmonologists with >1000 bronchoscopies each (expert). Participants were tested in 4 tasks (1: scope manipulation, 2: guided anatomic navigation, 3: airway anatomy, and 4: lymph node anatomy). Participants were scored and surveyed on their impressions of simulation. The means and Kruskal-Wallis test among groups were compared by task item (P<0.05).

Results: There were statistically significant differences in mean ranks among groups for tasks 1 and 3. For task 1, final score, time, mid-lumen time, and wall hits were discriminative (P=0.006, 0.006, 0.012, and 0.014, respectively). For task 3, time, bronchial segments identified, bronchial segments incorrectly identified, and bronchial segments skipped were discriminative (P=0.04, 0.012, 0.013, and 0.013, respectively). There was no statistically significant difference for task 2 and task 4. All participants agreed that simulation training is helpful and should be incorporated into bronchoscopic training.

Conclusions: The simulator demonstrated validity in differentiating skill in scope manipulation and airway anatomy, but did not discriminate skill levels in anatomic orientation or identification of lymph nodes. Bronchoscopy simulation was viewed as helpful by all levels and should be considered before performance on patients.


Objective structured clinical examination in basic thoracic ultrasound: a European study of validity evidence Pietersen PI et al, BMC Pulmonary Medicine, 2023

Background: Basic thoracic ultrasound is being used more frequently by clinicians in several settings due to its high diagnostic accuracy for many common causes of respiratory failure and dyspnoea. However, ultrasound examinations are operator-dependent, and sufficient competences are needed to obtain high sensitivity and specificity of examinations. Additionally, it is crucial for ultrasound operators to perceive the competence to interpret the images and integrate them into the patient history and other examinations. This study aims to explore and gather validity evidence for an objective structured clinical examination test of basic thoracic ultrasound competences and establish a pass/fail score.

Methods An expert panel created the test which included two theoretical and five practical stations representing cases with different diagnoses that cause respiratory symptoms and which are possible to establish by basic thoracic ultrasound. Twenty-five participants with different levels of experience in basic thoracic ultrasound completed the test. Data of the test scores were used for item analysis, and exploring validity evidence was done according to Messick’s framework which is recommended. The contrasting groups’ standard-setting method was used to establish a pass/fail score.

Results The summarised internal consistency reliability was high with a Cronbach’s alpha of 0.87. The novice group (n = 4) had a mean test score of 42 ± 10.1 points, the intermediate group (n = 8) scored 79.1 ± 8.1 points, and the experienced group (n = 13) 89.0 ± 6.2 points (one-way ANOVA, p < 0.001). A pass/fail score of 71 points was thus derived (maximum test score = 105 points).

Conclusion We developed a test for the assessment of clinical competences in basic thoracic ultrasound with solid validity evidence, and a pass/fail standard with no false positives or false negatives.


Construct Validity of a Virtual Reality Simulator for Surgical Training in Knee Arthroscopy Palet MJ et al, Cureus, May 25, 2021

Objective: Surgical techniques are learned gradually throughout an orthopedic residency. Training on real patients carries drawbacks such as limited access and elevated risk. Alternatively, surgical simulation allows residents to practice in a safe environment with greater access to standardized surgical tasks. Virtual reality simulators display images inside an artificial joint, often providing real-time haptic feedback to allow for realistic interaction. The objective of this study was to evaluate the construct validity of a virtual reality simulator for knee arthroscopy by analyzing the capacity of system parameters to distinguish between expert and novice surgeons.

Design: This comparative cross-sectional study contrasts the automated performance reports for novice and expert orthopedic surgeons after executing surgical tasks on the ARTHRO Mentor virtual reality simulator. Setting Surgical simulation center at the University of Chile Clinical Hospital, Santiago, Chile.

Participants: The novice group consisted of 20 second-year orthopedic and traumatology residents at the University of Chile School of Medicine. The expert group consisted of 10 experienced arthroscopic surgeons. All participants carried out standardized tasks in the knee arthroscopy virtual reality simulator. The median performance scores of the two groups were compared, and multivariate logistic regression was performed to assess the capacity of the system to discriminate between the two groups.

Results: Median performance on the vast majority of surgical tasks was superior for the expert group. The expert group had performance values equal to or higher than the novice group on 43 of the 44 variables recorded for the basic tasks and 74 of the 75 advanced task variables. The multivariate logistic regression analysis discriminated expert from novice users with 100% accuracy.

Conclusion: The virtual reality simulator for knee arthroscopy showed good construct validity, with performance metrics accurately discriminating between expert and novice users.


Effectiveness of Simulation-Based Training on Transesophageal Echocardiography Learning Pezel et al, JAMA Cardiol. Published online January 11, 2023.

Objective:  To assess the effectiveness of simulation-based teaching vs traditional teaching of TEE knowledge and skills of cardiology fellows.

Design, Setting, and Participants:  Between November 2020 and November 2021, all consecutive cardiology fellows inexperienced in TEE from 42 French university centers were randomized (1:1; n = 324) into 2 groups with or without simulation support.

Main Outcomes and Measures:  The co-primary outcomes were the scores in the final theoretical and practical tests 3 months after the training. TEE duration and the fellows’ self-assessment of their proficiency were also assessed.

Results:  While the theoretical and practical test scores were similar between the 2 groups (324 participants; 62.6% male; mean age, 26.4 years) before the training (33.0 [SD, 16.3] points vs 32.5 [SD, 18.5] points; P = .80 and 44.2 [SD, 25.5] points vs 46.1 [SD, 26.1] points; P = .51, respectively), the fellows in the simulation group (n = 162; 50%) displayed higher theoretical test and practical test scores after the training than those in the traditional group (n = 162; 50%) (47.2% [SD, 15.6%] vs 38.3% [SD, 19.8%]; P < .001 and 74.5% [SD, 17.7%] vs 59.0% [SD, 25.1%]; P < .001, respectively). Subgroup analyses showed that the effectiveness of the simulation training was even greater when performed at the beginning of the fellowship (ie, 2 years or less of training) (theoretical test: an increase of 11.9 points; 95% CI, 7.2-16.7 vs an increase of 4.25 points; 95% CI, −1.05 to 9.5; P = .03; practical test: an increase of 24.9 points; 95% CI, 18.5-31.0 vs an increase of 10.1 points; 95% CI, 3.9-16.0; P < .001). After the training, the duration to perform a complete TEE was significantly lower in the simulation group than in the traditional group ( 8.3 [SD, 1.4] minutes vs 9.4 [SD, 1.2] minutes; P < .001, respectively). Additionally, fellows in the simulation group felt more ready and more confident about performing a TEE alone after the training (mean score, 3.0; 95% CI, 2.9-3.2 vs mean score, 1.7; 95% CI, 1.4-1.9; P < .001 and mean score, 3.3; 95% CI, 3.1-3.5 vs mean score, 2.4; 95% CI, 2.1-2.6; P < .001, respectively).

Conclusions and Relevance:  Simulation-based teaching of TEE showed a significant improvement in the knowledge, skills, and self-assessment of proficiency of cardiology fellows, as well as a reduction in the amount of time needed to complete the examination. These results should encourage further investigation of clinical performance and patient benefits of TEE simulation training.


Distribution of Innate Psychomotor Skills Recognized as Important for Surgical Specialization in Unconditioned Medical Undergraduates Moglia A. et al. Surg Endosc. 2018

Andrea Moglia, Luca Morelli, Vincenzo Ferrari, Mauro Ferrari, Franco Mosca, Alfred Cuschieri


Background: There is an increasing interest for a test assessing objectively the innate aptitude for surgery as a craft specialty to complement the current selection process of surgical residents. The aim of this study was to quantify the size of individuals with high, average, and low level of innate psychomotor skills among medical students.

Methods: A volunteer sample of 155 medical students, without prior experience with surgical simulator, executed five tasks at a virtual simulator for robot-assisted surgery. They had to reach proficiency twice consecutively in each before moving to the next one. A weighting based on time and number of attempts needed to reach proficiency was assigned to each task.

Results: Nine students (5.8%) out of 155 significantly outperformed all the others on median (i.q.r.) weighted time [44.7 (42.2–47.3) min vs. 98.5 (70.8–131.8) min, < 0.001], and number of attempts to reach proficiency [14 (12–15) vs. 23 (19–32.75), < 0.001). Seventeen students (11.0%) scored significantly much worse than the rest on median weighted time [202.2 (182.5–221.0) min vs. 84.3 (65.7–114.4) min, < 0.001], and number of attempts [42 (40–48) vs. 22 (17.25–28), < 0.001]. Low correlation between simulator scores and extracurricular activities, like videogames and musical instruments, was found.

Conclusions: The test successfully identified two groups straddling the large cohort with average innate aptitude for psychomotor skills: (i) innately gifted and (ii) with scarce level. Hence, exercises on a virtual simulator are a valid test of innate manual dexterity and can be considered to complement the selection process for a surgical training program, primarily to identify individuals with low innate aptitude for surgery and advise them to consider specialization in other (non-craft) medical specialties.


Laparoscopic Warm-up Exercises Improve Performance of Senior-Level Trainees During Laparoscopic Renal Surgery Lee et al, Journal of Endourology 2012

Background and Purpose: Surgery is a high-stakes “performance.” Yet, unlike athletes or musicians, surgeons do not engage in routine “warm-up” exercises before “performing” in the operating room. We study the impact of a preoperative warm-up exercise routine (POWER) on surgeon performance during laparoscopic surgery.

Materials and Methods: Serving as their own controls, each subject performed two pairs of laparoscopic cases, each pair consisting of one case with POWER (+POWER) and one without (–POWER). Subjects were randomly assigned to +POWER or −POWER for the initial case of each pairing, and all cases were performed ≥1 week apart. POWER consisted of completing an electrocautery skill task on a virtual reality simulator and 15 minutes of laparoscopic suturing and knot tying in a pelvic box trainer. For each case, cognitive, psychomotor, and technical performance data were collected during two different tasks: mobilization of the colon (MC) and intracorporeal suturing and knot tying (iSKT). Statistical analysis was performed using SYSTAT v11.0.

Results: A total of 28 study cases (14+POWER, 14−POWER) were performed by seven different subjects. Cognitive and psychomotor performance (attention, distraction, workload, spatial reasoning, movement smoothness, posture stability) were found to be significantly better in the +POWER group (P≤0.05) and technical performance, as scored by two blinded laparoscopic experts, was found to be better in the +POWER group for MC (P=0.04) but not iSKT (P=0.92). Technical scores demonstrated excellent reliability using our assessment tool (Cronbach ∝=0.88). Subject performance during POWER was also found to correlate with intraoperative performance scores.

Conclusions: Urologic trainees who perform a POWER approximately 1 hour before laparoscopic renal surgery demonstrate improved cognitive, psychomotor, and technical performance.


Development of a structured virtual reality curriculum for laparoscopic appendicectomy Sinitsky et al, The American Journal of Surgery 2019

Background: Laparoscopic appendicectomy (LA) is a common surgical emergency procedure mainly performed by trainees. The aim was to develop a step-wise structured virtual reality (VR) curriculum for LA to allow junior surgeons to hone their skills in a safe and controlled environment.

Methods: A prospective randomized study was designed using a high-fidelity VR simulator. Thirty-five novices and 25 experts participated in the assessment and their performances were compared to assess construct validity. Learning curve analysis was performed.

Results: Five of the psychomotor tasks and all appendicectomy tasks showed construct validity. Learning was demonstrated in the majority of construct-valid tasks. A novel goal-directed VR curriculum for LA was constructed. Conclusions: A step-wise structured VR curriculum for LA is proposed with a framework which includes computer generated metrics and supports deliberate practice, spacing intervals, human instruction/feedback and assessment. Future study should test the feasibility of its implementation and transferability of acquired skill.


Predictive Validity of a Training Protocol Using a Robotic Surgery Simulator Culligan P. et al. Urogynecology, 2014

Background:   Robotic surgery simulation may provide a way for surgeons to acquire specific robotic surgical skills without practicing on live patients.

Methods:   Five robotic surgery experts performed 10 simulator skills to the best of their ability, and thus, established expert benchmarks for all parameters of these skills. A group of credentialed gynecologicsurgeons naive to robotics practiced the simulator skills until theywere able to perform each one as well as our experts. Within a weekof doing so, they completed robotic pig laboratory training, after whichthey performed supracervical hysterectomies as their first-ever livehuman robotic surgery. Time, blood loss, and blinded assessments ofsurgical skill were compared among the experts, novices, and a groupof control surgeons who had robotic privileges but no simulator expo-sure. Sample size estimates called for 11 robotic novices to achieve90% power to detect a 1 SD difference between operative times ofexperts and novices (> = 0.05).

Results:   Fourteen novice surgeons completed the studyVspendingan average of 20 hours (range, 9.7Y38.2 hours) in the simulation lab-oratory to pass the expert protocol. The mean operative times for the ex-pert and novices were 20.2 (2.3) and 21.7 (3.3) minutes, respectively(P = 0.12; 95% confidence interval, j1.7 to 4.7), whereas the mean timefor control surgeons was 30.9 (0.6) minutes (P G 0.0001; 95% confi-dence interval, 6.3Y12.3). Comparisons of estimated blood loss (EBL) andblinded video assessment of skill yielded similar differences betweengroups.

Conclusions:   Completing this protocol of robotic simulator skills translated to expert-level surgical times during live human surgery. As such, we have established predictive validity of this protocol.


A Systematic Review of Simulation-Based Training in Vascular Surgery Haiser A. et Al., J Surg Res., 2022

Introduction: Recent advancements in surgical technology, reduced working hours, and training opportunities exacerbated by the COVID-19 pandemic have led to an increase in simulation-based training. Furthermore, a rise in endovascular procedures has led to a requirement for high-fidelity simulators that offer comprehensive feedback. This review aims to identify vascular surgery simulation models and assess their validity and levels of effectiveness (LoE) for each model in order to successfully implement them into current training curricula.

Methods: PubMed and EMBASE were searched on January 1, 2021, for full-text English studies on vascular surgery simulators. Eligible articles were given validity ratings based on Messick’s modern concept of validity alongside an LoE score according to McGaghie’s translational outcomes.

Results: Overall 76 eligible articles validated 34 vascular surgery simulators and training courses for open and endovascular procedures. High validity ratings were achieved across studies for: content , response processes , the internal structure , relations to other variables, and consequences. Only seven studies achieved an LoE greater than 3/5. Overall, ANGIO Mentor was the most highly validated and effective simulator and was the only simulator to achieve an LoE of 5/5.

Conclusions: Simulation-based training in vascular surgery is a continuously developing field with exciting future prospects, demonstrated by the vast number of models and training courses. To effectively integrate simulation models into current vascular surgery curricula and assessments, there is a need for studies to look at trainee skill retention over a longer period of time. A more detailed discussion on cost-effectiveness is also needed.


A Proficiency-Based Stepwise Endovascular Curricular Training (PROSPECT) Program Enhances Operative Performance in Real Life: A Randomised Controlled Trial Maertens H et al., Eur J Vasc Endovasc Surg 2017

Objectives: Healthcare evolution requires optimisation of surgical training to provide safe patient care. Operating
room performance after completion of proficiency based training in vascular surgery has not been investigated.

Design: A randomised controlled trial evaluated the impact of a Proficiency based Stepwise Endovascular
Curricular Training program (PROSPECT) on the acquisition of endovascular skills and the transferability of these
skills to real life interventions.

Materials: All subjects performed two endovascular interventions treating patients with symptomatic iliac and/or
superficial femoral artery stenosis under supervision. Primary outcomes were technical performances (Global Rating
Scale [GRS]; Examiner Checklist), operative metrics, and patient outcomes, adjusted for case difficulty and trainee
experience. Secondary outcomes included knowledge and technical performance after 6 weeks and 3 months.

Methods: Thirty-two general surgical trainees were randomised into three groups. Besides traditional training, the first group (n ¼ 11) received e-learning and simulation training (PROSPECT), the second group (n ¼ 10) only had access to e-learning, while controls (n ¼ 11) did not receive supplementary training.

Results: Twenty-nine trainees (3 dropouts) performed 58 procedures. Trainees who completed PROSPECT showed
superior technical performance (GRS 39.36 +/- 2.05; Checklist 63.51 +/- 3.18) in real life with significantly fewer
supervisor takeovers compared with trainees receiving e-learning alone (GRS 28.42 +/- 2.15; p ¼ .001; Checklist
53.63 +/- 3.34; p ¼ .027) or traditional education (GRS 23.09 +/- 2.18; p ¼ .001; Checklist 38.72 +/- 3.38; p ¼ .001).
Supervisors felt more confident in allowing PROSPECT trained physicians to perform basic (p ¼ .006) and complex
(p ¼ .003) procedures. No differences were detected in procedural parameters (such as fluoroscopy time, DAP,
procedure time, etc.) or complications. Proficiency levels were maintained up to 3 months.

Conclusions: A structured, stepwise, proficiency-based endovascular curriculum including e-learning and
simulation-based training should be integrated early into training programs to enhance trainee performance.


Validation of a High-Fidelity Electrophysiology Simulator and Development of a Proficiency-Based Simulator Training Program Ullah W. et al., Simul Healthc. 2017

Introduction: Simulator training can potentially provide high-intensity training in electrophysiology (EP) without compromising patient safety. We assessed the construct validity of a novel EP simulator (ANGIO Mentor; Simbionix) and developed proficiency-based scores for clinical EP simulator training.

Methods: Two European training centers participated. Participants were grouped on the basis of years of EP experience and (for a subset) subjectively scored clinical catheter manipulation skills. Each participant attempted the simulator’s 5 modules 3 times. These modules focus on catheter manipulation in 3-dimensional models, ranging from geometric shapes to fluoroscopic contracting cardiac models, with performance scored by the system on the basis of attainment of preset targets. Using these scores, targets were formulated for basic EP training.

Results: Twenty-eight participants were included (13 with subjectively scored catheter manipulation). Scores for participants with less than 1-year EP experience (group 1) were significantly lower for each of the attempts at the modules (P < 0.002). For group 1 only, scores improved with subsequent attempts (P < 0.005). In 4 of the 5 modules, scores of group 1 were significantly lower than the more experienced groups (P < 0.0005). Participants with subjectively scored above-average catheter manipulation skills also had higher scores in 4 of the 5 modules (P < 0.05). Target scores for a proficiency-based training program were generated from the median scores for each module for those with 1-year experience or more.

Conclusions: Scores attained in the simulator can distinguish those with less than 1-year EP experience and those with above-average catheter manipulation skills. Consequently, target scores have been generated for a proficiency-based training program.


Objective structured clinical examination in basic thoracic ultrasound: a European study of validity evidence Pietersen P. et a;. BMC Pulmonary Medicine

Background:  Basic thoracic ultrasound is being used more frequently by clinicians in several settings due to its high diagnostic accuracy for many common causes of respiratory failure and dyspnoea. However, ultrasound examinations are operator-dependent, and sufficient competences are needed to obtain high sensitivity and specificity of
examinations. Additionally, it is crucial for ultrasound operators to perceive the competence to interpret the images and integrate them into the patient history and other examinations. This study aims to explore and gather validity evidence for an objective structured clinical examination test of basic thoracic ultrasound competences and establish
a pass/fail score.

Methods: An expert panel created the test which included two theoretical and five practical stations representing cases with different diagnoses that cause respiratory symptoms and which are possible to establish by basic thoracic ultrasound. Twenty-five participants with different levels of experience in basic thoracic ultrasound completed
the test. Data of the test scores were used for item analysis, and exploring validity evidence was done according to Messick’s framework which is recommended. The contrasting groups’ standard setting method was used to establish a pass/fail score.

Results:  The summarised internal consistency reliability was high with a Cronbach’s alpha of 0.87. The novice group (n = 4) had a mean test score of 42 ± 10.1 points, the intermediate group (n = 8) scored 79.1 ± 8.1 points, and the experienced group (n = 13) 89.0 ± 6.2 points (one-way ANOVA, p < 0.001). A pass/fail score of 71 points was thus
derived (maximum test score = 105 points).

Conclusion: We developed a test for the assessment of clinical competences in basic thoracic ultrasound with solid validity evidence, and a pass/fail standard with no false positives or false negatives.


Effect of virtual reality training on laparoscopic surgery: randomised controlled trial Christian R Larsen et al, British Medical Journal 2009

Objective: To assess the effect of virtual reality training on an actual laparoscopic operation.

Setting and Participants: Seven gynaecological departments in the Zeeland region of Denmark. 24 first and second year registrars specialising in gynaecology and obstetrics.

Interventions: Proficiency based virtual reality simulator training in laparoscopic salpingectomy and standard clinical education (controls).

Main outcome measure: The main outcome measure was technical performance assessed by two independent observers blinded to trainee and training status using a previously validated general and task specific rating scale. The secondary outcome measure was operation time in minutes.

Results: The simulator trained group (n=11) reached a median total score of 33 points (interquartile range 32-36 points), equivalent to the experience gained after 20-50 laparoscopic procedures, whereas the control group (n=10) reached a median total score of 23 (22-27) points, equivalent to the experience gained from fewer than five procedures (P<0.001). The median total operation time in the simulator trained group was 12 minutes (interquartile range 10-14 minutes) and in the control group was 24 (20-29) minutes (P<0.001). The observers’ inter-rater
agreement was 0.79.

Conclusion: Skills in laparoscopic surgery can be increased in a clinically relevant manner using proficiency based virtual reality simulator training. The performance level of novices was increased to that of intermediately experienced laparoscopists and operation time was halved. Simulator training should be considered before trainees carry out laparoscopic procedures.


Proficiency-based virtual reality training significantly reduces the error rate for residents during their first 10 laparoscopic cholecystectomies Ahlberg et al, American Journal of Surgery 2007

Background: Virtual reality (VR) training has been shown previously to improve intraoperative performance during part of a laparoscopic cholecystectomy. The aim of this study was to assess the effect of proficiency-based VR training on the outcome of the first 10 entire cholecystectomies performed by novices.

Methods: Thirteen laparoscopically inexperienced residents were randomized to either (1) VR training until a predefined expert level of performance was reached, or (2) the control group.  Videotapes of each resident’s first 10 procedures were reviewed independently in a blinded fashion and scored for predefined errors.

Results: The VR-trained group consistently made significantly fewer errors (P = .0037). On the other hand, residents in the control group made, on average, 3 times as many errors and used 58% longer surgical time. Conclusions: The results of this study show that training on the VR simulator to a level of proficiency significantly improves intraoperative performance during a resident’s first 10 laparoscopic cholecystectomies.


Predictors of laparoscopic simulation performance among practicing obstetrician gynecologists Shyama Mathews et al, Am J Obstet Gynecol 2017

Background: While simulation training has been established as an effective method for improving laparoscopic surgical performance in surgical residents, few studies have focused on its use for attending surgeons, particularly in obstetrics and gynecology. Surgical simulation may have a role in improving and maintaining proficiency in the operating room for practicing obstetrician gynecologists.

Objective: We sought to determine if parameters of performance for validated laparoscopic virtual simulation tasks correlate with surgical volume and characteristics of practicing obstetricians and gynecologists.

Study Design: All gynecologists with laparoscopic privileges (n ¼ 347) from 5 academic medical centers in New York City were required to complete a laparoscopic surgery simulation assessment. The physicians took a presimulation survey gathering physician self-reported characteristics and then performed 3 basic skills tasks (enforced peg transfer, lifting/grasping, and cutting) on the LapSim virtual reality laparoscopic simulator (Surgical Science Ltd, Gothenburg, Sweden). The association between simulation outcome scores (time, efficiency, and
errors) and self-rated clinical skills measures (self-rated laparoscopic skill score or surgical volume category) were examined with regression models.

Results: The average number of laparoscopic procedures per month was a significant predictor of total time on all 3 tasks (P ¼ .001 for peg transfer; P ¼ .041 for lifting and grasping; P < .001 for cutting). Average monthly laparoscopic surgical volume was a significant predictor of 2 efficiency scores in peg transfer, and all 4 efficiency scores in cutting (P¼ .001 to P ¼ .015). Surgical volume was a significant predictor of errors in lifting/grasping and cutting (P < .001 for both). Self-rated laparoscopic skill level was a significant predictor of total time in all 3 tasks (P < .0001 for peg transfer; P ¼ .009 for lifting and grasping; P < .001 for cutting) and a significant predictor of nearly all efficiency scores and errors scores in all 3 tasks.

Conclusion: In addition to total time, there was at least 1 other objective performance measure that significantly correlated with surgical volume for each of the 3 tasks. Higher-volume physicians and those with fellowship training were more confident in their laparoscopic skills. By determining simulation performance as it correlates to active physician practice, further studies may help assess skill and individualize training to maintain skill levels as case volumes fluctuate.


Prospective Cohort Study of Haptic Virtual Reality Laparoscopic Appendicectomy Learning Curve Trajectory Chris Brown et al, J LapEndo & Advanced Surg Tech 2019

Background: Simulation training is strongly advocated by 24/7 risk-rich professions because swift learning curve inflection point attainment delivers earlier competence; the left-shift effect. The aim of this study was to determine the value of haptic laparoscopic virtual reality simulation, by iterative benchmark exercise (n = 8), before simulated laparoscopic appendicectomy (SLA); the hypothesis was that favorable benchmark learning curve trajectories would be associated with improved SLA competence when compared with consultant expert performance.

Methods: A 28-trainee cohort completed 1349 Laparoscopic Haptic Virtual Reality Skills (LHVRS) tasks, during which 19 ergonomic variables were assessed by virtual interface, including force feedback (, before 153 SLAs. Primary outcome measure was SLA composite competence score related to six consultant trainer experts.

Results: Of the eight LHVRS tasks, the three with the steepest learning curve trajectories correlated with better median overall SLA competence scores, namely tissue grasping/lifting (rho = 0.362, P = .049), fine dissection (rho = 0.388, P = .028), and camera navigation (rho = 0.518, P = .007); fine dissection was the only haptic laparoscopic virtual reality simulation task that predicted a SLA score within a Youden index defined, 70% of the consultant expert level (area under curve [AUC] = 0.803, P = .028). A significant SLA learning curve emerged, with a learning curve trajectory inflection point at the fourth SLA attempt (first SLA 30.5% versus fourth SLA score 76.0%, gradient 76°, P = .010).

Conclusion: Learning curve trajectory can be measured, influenced, and accelerated significantly; a pronounced left-shift effect, with translational potential for enhanced shorter training time and improved patient safety.


Controlled Prospective Study on Ultrasound Simulation Training in Fetal Echocardiography: FESIM II Archives of Gynecology and Obstetrics, Janzing P. et al

Purpose: To analyze the learning curves of ultrasound novices in fetal echocardiography during structured simulation-based ultrasound training (SIM-UT) including a virtual, randomly moving fetus.

Methods: 11 medical students with minimal (< 10 h) prior obstetric ultrasound experience underwent 12 h of structured fetal echocardiography SIM-UT in individual hands-on sessions during a 6-week training program. Their learning progress was assessed with standardized tests after 2, 4, and 6 weeks of SIM-UT. Participants were asked to obtain 11 fetal echocardiography standard planes (in accordance with ISUOG and AHA guidelines) as quickly as possible. All tests were carried out under real life, examination-like conditions on a healthy, randomly moving fetus. Subsequently, we analyzed the rate of correctly obtained images and the total time to completion (TTC). As reference groups, 10 Ob/Gyn physicians (median of 750 previously performed Ob/Gyn scans) and 10 fetal echocardiography experts (median of 15,000 previously performed Ob/Gyn scans) were examined with the same standardized tests.

Results: The students showed a consistent and steady improvement of their ultrasound performance during the training program. After 2 weeks, they were able to obtain > 95% of the standard planes correctly. After 6 weeks, they were significantly faster than the physician group (p < 0.001) and no longer significantly slower than the expert group (p = 0.944).

Conclusion: SIM-UT is highly effective to learn fetal echocardiography. Regarding the acquisition of the AHA/ISUOG fetal echocardiography standard planes, the students were able to reach the same skill level as the expert group within 6 weeks.


Virtual Reality Simulation Of Neuroendovascular Intervention Improves Procedure Speed In A Cohort Of Trainees Dardick J. et al, Surgical Neurology Int 2019

Background: Realistic virtual reality (VR) simulators have greatly expanded the tools available for training surgeons and interventionalists. While this technology is effective in improving performance in many fields, it has never been evaluated for neuroendovascular procedures. This study aims to determine whether VR is an effective tool for improving neuroendovascular skill among trainees.

Methods: Trainees performed two VR revascularizations of a right-sided middle cerebral artery (MCA) thrombosis and their times to procedural benchmarks (time to enter internal carotid artery [ICA], traverse clot, and complete procedure) were compared. To determine whether the improvement was case specific, trainees with less procedural exposure were timed during VR left-sided ICA (LICA) aneurysm coiling before or after performing MCA thrombectomy simulations. To determine the value of observing simulations, medical students were timed during the right MCA revascularization simulations after watching other VR procedures.

Results: Trainees significantly improved their time to every procedural benchmark during their second MCA revascularization (mean decrease = 1.08, 1.57, and 2.24 min; P = 0.0072, 0.0466, and 0.0230). In addition, time required to access the LICA during aneurysm coiling was shortened by 0.77 min for each previous VR right MCA revascularization performed (P = 0.0176; r2 = 0.71). Finally, medical students’ MCA revascularization simulation times improved by 0.87 min for each prior simulation viewed (P r2 = 0.96).

Conclusion: Both performance and viewing of simulated procedures produced significant decreases in time to reach neuroendovascular procedural benchmarks. These data show that VR simulation is a valuable tool for improving trainee skill in neuroendovascular procedures.


Effectiveness of the URO Mentor Virtual Reality Simulator in the Skill Acquisition of Flexible Cystoscopy Zhang Y., Chinese Medical Journal (Engl). 2013

Background: Virtual reality (VR) has been recognized as a useful modality in the training of surgical skills. With respect to basic endoscopic skill training of urology, we sought to investigate the effectiveness of the UroMentor(TM) virtual reality simulator (VRS) in the skill acquisition of flexible cystoscopy.

Methods: Urologists familiar with rigid cystoscopy procedures were selected to take part in a virtual training course of flexible cystoscopy. Changes in total operating time, frequency of injury, number of digital markers inside the bladder, and the global rating scale (GRS) scores were assessed following eight repeated training sessions on the UroMentor(TM).

Results: Eighteen urologists voluntarily took part in the study. Total operating time was significantly lower after eight sessions of training by comparison ((111 ± 10) seconds and (511 ± 67) seconds, respectively; P < 0.001). Additionally, the frequency of injury decreased with training from (12 ± 2) times to (5 ± 1) times (P < 0.001), while the number of digital markers observed increased from 9 ± 0 to 10 ± 1 (P = 0.005). Finally, training with the UroMentor(TM) resulted in a GRS increase from (1.3 ± 0.2) points to (3.9 ± 0.2) points (P < 0.001).

Conclusion: The VRS UroMentor(TM) can improve urologists’ ability to perform flexible cystoscopy and could be used as an effective training tool for trainees.


Assessment of Urology Postgraduate Trainees’ Competencies in Flexible Ureteroscopic Stone Extraction Aloosh M. et al., Canadian Urological Associaiton Journal, 2018

Objectives: This study aims to assess the impact of a virtual reality trainer in improving percutaneous renal access skills of urological trainees.

Methods: A total of 36 urology trainees participated in this prospective study. Initially, they were taken through the exercise of gaining access to the lower pole calyceal system and introducing a guidewire down the ureter. Trainees’ performance was then assessed by virtual reality-derived parameters of the simulator at baseline and after 2 h of training.

Results: Participants who underwent training with the simulator demonstrated significant improvement in several parameters compared to their baseline performance. There was a statistically significant correlation between total time to perform the procedure and time of radiation exposure, radiation dose and correct calyx puncture (p < 0.01). Trainees needed a mean of 15.8 min from skin puncture to correct guidewire placement into the pelvicalyceal system before and 6.49 min following training.

Conclusions: We found percutaneous renal access skills of trainees improve significantly on a number of parameters as a result of training on the PERC Mentor TM VR simulator. Such simulated training has the potential to decrease the risks and complications associated with the early stages of the learning curve when training for percutaneous renal access in patients.


Assessment of Esophagogastroduodenoscopy Skills on Simulators Before Real-Life Performance Nielsen AB et al., Endosc Int Open. 2022

Background and Study Objectives: Operator competency is essential for esophagogastroduodenoscopy (EGD) quality, which makes appropriate training with a final test important. The aims of this study were to develop a test for assessing skills in performing EGD, gather validity evidence for the test, and establish a credible pass/fail score.

Methods: An expert panel developed a practical test using the Simbionix GI Mentor II simulator (3 D Systems) and an EGD phantom (OGI 4, CLA Medical) with a diagnostic (DP) and a technical skills part (TSP) for a prospective validation study. During the test a supervisor measured: 1) total time; 2) degree of mucosal visualization; and 3) landmarks and pathology identification. The contrasting groups standard setting method was used to establish a pass/fail score.

Results: We included 15 novices (N), 10 intermediates (I), and 10 experienced endoscopists (E). The internal structure was high with a Cronbach’s alpha of 0.76 for TSP time consumption and 0.74 for the identification of landmarks.

Mean total times, in minutes, for the DP were N 15.7, I 11.3, and E 7.0, and for TSP., they were N 7.9, I 8.9, and E 2.9. The total numbers of identified landmarks were N 26, I 41, and E 48. Mean visualization percentages were N 80, I 71, and E 71. A pass/fail standard was established requiring identification of all landmarks and performance of the TSP in < 5 minutes. All experienced endoscopists passed, while none of the endoscopists in the other categories did.

Conclusions: We established a test that can distinguish between participants with different competencies. This enables an objective and evidence-based approach to assessment of competencies in EGD.


Providing of a Virtual Simulator Perineal Anatomy (Pelvic Mentor®) In Learning Pelvic Perineology: Results of a Preliminary Study Legendre G. et al, J Gynecol Obstet Biol Reprod (Paris) 2015

Introduction: Medical and surgical simulation is in high demand. It is widely used in North America as a method of education and training of medical students and surgical residents. Learning anatomy and vaginal surgery are based on palpation recognition of different structures. The absence of visual control of actions learners is a limiting factor for the reproducibility of surgical techniques prolapse and urinary incontinenence. However, this reproducibility is the only guarantee of success and safety of these minimally invasive surgeries.

Methods: We evaluated the contribution of an educational module perineal anatomy using a system combining anatomic mannequin and a computerized 3D virtual simulator (Pelvic Mentor®, Simbionix) in the knowledge of pelvic-perineal anatomical structures for eight residents of obstetrics and gynecology hospitals in Paris.

Results: The self-study training module has led to substantial improvements in internal rating with a proportion of structures recognized from 31.25 to 87.5 % (P<0.001) for the front compartment and 20 to 85 % (P<0.001) for the posterior compartment.

Conclusion: The preliminary results suggest that the 3D virtual simulator enhances and facilitates learning the anatomy of the pelvic floor.


Early Implementation of Fundamentals of Endoscopic Surgery Training Using a Simulation-Based Mastery Learning Curriculum Dyke C. et al., Surgery, 2021

Background: After the American Board of Surgery announcement of the Flexible Endoscopy Curriculum requirement in 2014, we implemented a dedicated endoscopy rotation at the post graduate year (PGY)2 level including a simulation curriculum for Fundamentals of Endoscopic Surgery skills. Here we evaluate the outcomes of this implementation.

Methods: Beginning in 2015, we developed a clinical endoscopy and simulation-based rotation to prepare for Fundamentals of Endoscopic Surgery testing. Originally, our curriculum was based on the published Texas Association of Surgical Skills Laboratories curriculum using the GI Mentor and transitioned to a mastery learning curriculum using the Endoscopy Training System in 2016. We evaluated the success of the curriculum in terms of first-time pass rates, training time required, and comparison to previously published benchmarks based on clinical experience.

Results: Since 2015, a total of 37 general surgery residents in our program were Fundamentals of Endoscopic Surgery tested (PGY2 = 24, PGY3 = 4, PGY5 = 9); 84% (31) completed the Endoscopy Training System curriculum. At the time of testing, 73% (27) had performed <25 esophagogastroduodenoscopies, and 46% had performed <25 colonoscopies. Ninety-two percent (34) spent 10 hours or less completing the curriculum. The first-time pass rate for those completing the Endoscopy Training System curriculum was 97% vs 67% for those not completing the Endoscopy Training System curriculum (P = .01). For residents completing the Endoscopy Training System curriculum, total Fundamentals of Endoscopic Surgery scores were discernibly higher (472 vs 389, P < .01), as were 3/5 task scores (Nav1 80 vs 67, P = .02; Loop2 36 vs 8, P = .02; Retro3 89 vs 71, P = .02). Despite clinical inexperience (<25 esophagogastroduodenoscopies and <50 colonoscopies), PGY2s yielded a mean score of 454 and a pass rate of 92%. This was similar to PGY5s (427, 89%; P = .3) and compares to benchmark data of endoscopists with >300 cases.

Conclusion: Early implementation of flexible endoscopy training with a simulation-based curriculum results in Fundamentals of Endoscopic Surgery performance equal to a clinical experience not often gained during surgical residency. Often requiring <10 hours, this represents a fantastic return on investment for this training.


The Learning Curves of a Validated Virtual Reality Hip Arthroscopy Simulator Bartlett JD et al, Archives of Orthopaedic and Trauma Surgery, 2020

Introduction: Decreases in trainees’ working hours, coupled with evidence of worse outcomes when hip arthroscopies are performed by inexperienced surgeons, mandate an additional means of training. Though virtual reality simulation has been adopted by other surgical specialties, its slow uptake in arthroscopic training is due to a lack of evidence as to its benefits. These benefits can be demonstrated through learning curves associated with simulator training—with practice reflecting increases in validated performance metrics.

Methods: Twenty-five medical students with no previous experience of hip arthroscopy completed seven weekly simulated arthroscopies of a healthy virtual hip joint using a 70° arthroscope in the supine position. Twelve targets were visualised within the central compartment, six via the anterior portal, three via the anterolateral portal and three via the posterolateral portal. Task duration, number of collisions (bone and soft-tissue), and distance travelled by arthroscope were measured by the simulator for every session of each student.

Results: Learning curves were demonstrated by the students, with improvements in time taken, number of collisions (bone and soft-tissue), collision length and efficiency of movement (all p<0.01). Improvements in time taken, efficiency of movement and number of collisions with soft-tissue were first seen in session 3 and improvements in all other parameters were
seen in session 4. No differences were found after session 5 for time taken and length of soft-tissue collision. No differences in number of collisions (bone and soft-tissue), length of collisions with bone, and efficiency of movement were found after session 6.

Conclusions: The results of this study demonstrate learning curves for a hip arthroscopy simulator, with significant improvements seen after three sessions. All performance metrics were found to improved, demonstrating sufficient visuo-haptic consistency within the virtual environment, enabling individuals to develop basic arthroscopic skills.


Arthroscopic Training Courses Improve Trainee Arthroscopy Skills: A Simulation-Based Prospective Trial Kevin D Martin, David P Patterson, Kenneth L Cameron, Arthroscopy 2016

Purpose: To evaluate the correlation between timed task performance on an arthroscopy shoulder simulator and participation in a standardized expert shoulder arthroscopy educational course.

Methods: Orthopaedic trainees were voluntarily recruited from over 25 residency programs throughout the United States and Canada. Each trainee was tested on arrival at the Arthroscopy Association of North America orthopaedic learning center on a virtual reality arthroscopy shoulder simulator, and his or her performance was objectively scored. Each trainee’s postgraduate year level was recorded, as was his or her experience in residency with shoulder arthroscopy as measured by Accreditation Council for Graduate Medical Education case-log totals. After the focused 4-day training curriculum consisting of didactics and cadaveric experience, each trainee was re-evaluated on the same simulator. Statistical analysis was performed to determine if participation in the course was associated with changes in simulation performance from before to after assessment.

Results: Forty-eight trainees completed the testing. On completion of the course, trainees showed significant improvements in all objective measures recorded by the simulator. Total probe distance needed to complete the task decreased by 42% (from 420.4 mm to 245.3 mm, P < .001), arthroscope tip distance traveled decreased by 59% (from 194.1 mm to
80.2 mm, P < .001), and time to completion decreased by 38% (from 66.8 seconds to 41.6 seconds, P < .001). Highly
significant improvements in all 3 measures suggest improved instrument handling, anatomic recognition, and
arthroscopy-related visual-spatial ability.

Conclusions: This study shows objective improvement in orthopaedic trainee basic arthroscopy skill and proficiency after a standardized 4-day arthroscopy training curriculum. The results validate the Arthroscopy Association of North America resident training course and its curriculum with objective evidence of benefit. Level of Evidence: Level III, prospective study of nonconsecutive participants.


Endovascular Training Using a Simulation Based Curriculum Is Less Expensive Than Training in the Hybrid ANGIO Suite Maertens H. et al., Eur J Vasc Endovasc Surg, 2018

Objective and Background: This study aimed to determine the cost-effectiveness of a Proficiency based Stepwise Endovascular Curricular Training (PROSPECT) program, including e-learning and hands on virtual reality simulation.

Methods: A prospective, single blinded, randomised controlled trial (RCT) was carried out to evaluate the impact of a PROSPECT training program on real life operative performance. Under supervision, all subjects performed two endovascular interventions on patients with symptomatic iliac and/or superficial femoral artery stenosis. Primary outcomes were technical performance (Global Rating Scale, Examiner Checklist), operative metrics, and patient outcomes, adjusted for case difficulty and the trainee’s experience. Additionally, an analysis of costs and savings related to implementation of this endovascular training program was performed. Thirty-two general surgery trainees were randomised into three groups: group 1 (n = 11) received e-learning and simulation training (PROSPECT program); group 2 (n = 10) only had access to e-learning; group 3 (n = 11) did not receive supplementary education besides clinical training. Developmental cost, implementation cost, training time cost, and the operational cost of PROSPECT were determined. Time spent studying and practicing was converted to indirect saving of operating time. The costs of logistics, faculty time supervising simulation sessions, and 30 day complication rates were registered. Sensitivity analysis was performed to assess the robustness of the results.

Results: Fifty-eight peripheral endovascular interventions, performed by 29 surgical trainees (three dropouts) were included in this RCT from October 2014 to February 2016. Annual costs from the perspective of the hospital were €6589 for curriculum design, €31,484 for implementation, and €1143 in operational costs. Per trainee, simulation-based training until proficiency cost €3806. In comparison, if endovascular proficiency levels were obtained with conventional training only, this may have cost €5001 per trainee.

Conclusion: Simulation based training in endovascular procedures may be cost saving, because training occurs outside the angiosuite. It is possible that cost savings are underestimated as, in contrast to the literature, prevented costs related to complications could not be defined.


Development And Validation Of Non-Guided Bladder-Neck And Neurovascular-Bundle Dissection Modules Of The Robotix-Mentor® Full-Procedure Robotic-Assisted Radical Prostatectomy Virtual Reality Simulation Ebbing J. et al, Int J Med Robot. 2021

Background: Full-procedure virtual reality (VR) simulator training in robotic-assisted radical prostatectomy (RARP) is a new tool in surgical education.

Methods: Description of the development of a VR RARP simulation model, (RobotiX-Mentor®) including non-guided bladder neck (ngBND) and neurovascular bundle dissection (ngNVBD) modules, and assessment of face, content, and construct validation of the ngBND and ngNVBD modules by robotic surgeons with different experience levels.

Results: Simulator and ngBND/ngNVBD modules were rated highly by all surgeons for realism and usability as a training tool. In the ngBND-task construct, validation was not achieved in task-specific performance metrics. In the ngNVBD, task-specific performance of the expert/intermediately experienced surgeons was significantly better than that of novices.

Conclusions: We proved face and content validity of simulator and both modules, and construct validity for generic metrics of the ngBND module and for generic and task-specific metrics of the ngNVBD module.


Does the Virtual Simulation Training Improve the Learning of Pelvic Ultrasound Performed in Gynecological Emergencies? G. Legendre et al., Ultrasound In Obstetrics & Gynecology, 2019

Objectives: To compare the contribution of training using virtual simulation with theoretical instruction alone in the learning of pelvic ultrasound for gynecological emergencies in non-experienced students.

Methods: A unicentre randomised controlled trial was conducted. The participants were students in the last year of medical studies during their internship in the obstetrics and gynecology department at the University Hospital Center of Angers. Twenty participants were randomised to a 1:1 ratio in the training group and control group of 10 students each. All participants received a two hours theoretical course on the physical basis of ultrasound, the use of probes, as well as the basic principles of an trans-vaginal ultrasound examination centreed on the different standardised images to be given to gynecologic emergencies. For the training group an additional 20-minute pelvic ultrasound training on a trans-vaginal virtual simulator was performed. The participants were evaluated on images taken from patients consulting our gynecological emergencies centre. The pictures evaluated were sagittal and coronal sections of the uterus, left and right ovarian sections and a Morrison pouch view. The primary outcome was based on picture quality scores described by Salomon et al. with the French Society for the Improvement of Ultrasound Practices (SFAPE) and by Popowski et al. for Standardisation Acute Female Echography (SAFE). Secondaries outcomes focused on the general ultrasound skills and the duration of acquisition of the pictures.

Results: The mean SFAPE and SAFE scores were significantly higher in the training group than in the control group (14.5 ± 3.1, p = 0.046 and 10.1 ± 2.08, p = 0.016, respectively).

Conclusions: After a very short training session, using virtual simulation, inexperienced students produce proper pictures for the management of gynecological emergencies and better quality ultrasound pictures than students trained only theoretically.


Developing and Gathering Validity Evidence for a Simulation-Based Test of Competencies in Lung Ultrasound Pietersen P.I. et al, Respiration 2019

Background: Clinical lung ultrasound (LUS) is a fast bedside diagnostic tool which can assist clinicians in decisions regarding the treatment and monitoring of patients with respiratory symptoms. LUS training and education differ widely, and is often done in a clinical setting, with potential risks for patients if decisions are made based on the wrong interpretations. No clear guidelines or recommendations for objective and standardized assessment of LUS skills exist, and those that do are often based on a fixed time-frame or an arbitrary number of examinations performed; this does not ensure adequate competencies.

Objectives: The study aimed to develop and gather validity evidence for a practical, simulation-based test in LUS.

Methods: Nine cases were developed in collaboration with 3D Systems Healthcare, Littleton, CO, USA, representing the most common diagnosis and sonographic findings in patients with respiratory symptoms. Thirty-six participants with different levels of competence in LUS, completed the test. The participants were divided into groups, i.e., novices, intermediates, and experienced, according to their experience with LUS, the number of examinations they had performed, and any research they had conducted. Their answers were used for item analyses.

Results: The intraclass correlation coefficient, Cronbachs’ α, was 0.69 summarized, and there was a statistically significant difference (p < 0.001) between the novices and the trained participants (intermediates and experienced). A pass/fail score of 16 points was calculated according to the contrasting-groups method.

Conclusion: We developed a test for the assessment of clinical competencies in LUS. The test proved solid validity evidence, and a pass/fail standard without any false-negatives, and only 2 explained false-positives.


Validation of a Novel Simulation-Based Test in Robot-Assisted Radical Prostatectomy Olsen RG, et al, Journal of Endourology 2021

Purpose: To investigate validity evidence for a simulator-based test in robot-assisted radical prostatectomy (RARP).

Materials and Methods: The test consisted of three modules on the RobotiX Mentor VR-simulator: Bladder Neck Dissection, Neurovascular Bundle Dissection, and Urethrovesical Anastomosis. Validity evidence was investigated by using Messick’s framework by including doctors with different RARP experience: novices (who had assisted for RARP), intermediates (robotic surgeons, but not RARP surgeons), or experienced (RARP surgeons). The simulator metrics were analyzed, and Cronbach’s alpha and generalizability theory were used to explore reliability. Intergroup comparisons were done with mixed-model, repeated measurement analysis of variance and the correlation between the number of robotic procedures and the mean test score were examined. A pass/fail score was established by using the contrasting groups’ method.

Results: Ten novices, 11 intermediates, and 6 experienced RARP surgeons were included. Six metrics could discriminate between groups and showed acceptable internal consistency reliability, Cronbach’s alpha = 0.49, p < 0.001. Test–retest reliability was 0.75, 0.85, and 0.90 for one, two, and three repetitions of tests, respectively. Six metrics were combined into a simulator score that could discriminate between all three groups, p = 0.002, p < 0.001, and p = 0.029 for novices vs intermediates, novices vs experienced, and intermediates vs experienced, respectively. Total number of robotic operations and the mean score of the three repetitions were significantly correlated, Pearson’s r = 0.74, p < 0.001.

Conclusion: This study provides validity evidence for a simulator-based test in RARP. We determined a pass/fail level that can be used to ensure competency before proceeding to supervised clinical training.


Training in Robotic Surgery: Initial Experience Using the Brazilian College of Surgeons Model Fernando de Barros et al, Rev Col Bras Cir. 2021

Objective: To present the initial experience of the first tier of surgeons trained in the new model of robotic surgery training proposed by the CBC.

Methods: We retrospectively collected data and information on training with the Da Vinci SI robotic system. The variables analyzed were, in the pre-clinical phase, time of completion of each step by surgeon and number of hours in the simulator, and in the clinical phase, operations carried out by the training group, number of surgeons who performed nine procedures in ninety days (“9 in 90”), time of docking, time of console, and results surgical.

Results: We interviewed 39 surgeons before training started; 20 (51.3%) reached the clinical phase. The average age of surgeons was 47.9 years (38-62). The average time between the first interview and the delivery of the online certificate was 64 days (15-133). The surgeons have made an average of 51h and 36 minutes of robot simulation (40-83 hours). The total number of cases in which the training surgeons participated as first assistant was 418, with an average of 20.9 per surgeon. The time of pre-clinical training had an average of 116 days (48-205).

Conclusion: The new model proposed had good acceptance by all surgeons trained and proved safe in the initial sample.


Effect of virtual reality training to enhance laparoscopic assistance skills Chen X et al., BMC Med. Edu., Jan 4, 2024


While laparoscopic assistance is often entrusted to less experienced individuals, such as residents, medical students, and operating room nurses, it is important to note that they typically receive little to no formal laparoscopic training. This deficiency can lead to poor visibility during minimally invasive surgery, thus increasing the risk of errors. Moreover, operating room nurses and medical students are currently not included as key users in structured laparoscopic training programs.


The aim of this study is to evaluate the laparoscopic skills of OR nurses, clinical medical postgraduate students, and residents before and after undergoing virtual reality training. Additionally, it aimed to compare the differences in the laparoscopic skills among different groups (OR nurses/Students/Residents) both before and after virtual reality training.


Operating room nurses, clinical medical postgraduate students and residents from a tertiary Grade A hospital in China in March 2022 were selected as participants. All participants were required to complete a laparoscopic simulation training course in 6 consecutive weeks. One task from each of the four training modules was selected as an evaluation indicator. A before-and-after self-control study was used to compare the basic laparoscopic skills of participants, and laparoscopic skill competency was compared between the groups of operating room nurses, clinical medical postgraduate students, and residents.


Twenty-seven operating room nurses, 31 clinical medical postgraduate students, and 16 residents were included. The training course scores for the navigation training module, task training module, coordination training module, and surgical skills training module between different groups (operating room nurses/clinical medical postgraduate/residents) before laparoscopic simulation training was statistically significant (p < 0.05). After laparoscopic simulation training, there was no statistically significant difference in the training course scores between the different groups. The surgical level scores before and after the training course were compared between the operating room nurses, clinical medical postgraduate students, and residents and showed significant increases (p < 0.05).


Our findings show a significant improvement in laparoscopic skills following virtual surgery simulation training across all participant groups. The integration of virtual surgery simulation technology in surgical training holds promise for bridging the gap in laparoscopic skill development among health care professionals.


Simulation-Based Bronchoscopy Training Kennedy CC, Maldonado F., Cook DA, CHEST July 2013, P183-192.

Background: Simulation-based bronchoscopy training is increasingly used, but effectiveness remains uncertain. We sought to perform a comprehensive synthesis of published work on simulation-based bronchoscopy training.

Methods: We searched MEDLINE, EMBASE, CINAHL, PsycINFO, ERIC, Web of Science, and Scopus for eligible articles through May 11, 2011. We included all original studies involving health professionals that evaluated, in comparison with no intervention or an alternative instructional approach, simulation-based training for flexible or rigid bronchoscopy. Study selection and data abstraction were performed independently and in duplicate. We pooled results using random effects meta-analysis.

Results: From an initial pool of 10,903 articles, we identified 17 studies evaluating simulation-based bronchoscopy training. In comparison with no intervention, simulation training was associated with large benefits on skills and behaviors (pooled effect size, 1.21 [95% CI, 0.82-1.60]; n=8 studies) and moderate benefits on time (0.62 [95% CI, 0.12-1.13]; n=7). In comparison with clinical instruction, behaviors with real patients showed nonsignificant effects favoring simulation for time (0.61 [95% CI, -1.47 to 2.69]) and process (0.33 [95% CI, -1.46 to 2.11]) outcomes (n=2 studies each), although variation in training time might account for these differences. Four studies compared alternate simulation-based training approaches. Inductive analysis to inform instructional design suggested that longer or more structured training is more effective, authentic clinical context adds value, and animal models and plastic part-task models may be superior to more costly virtual-reality simulators.

Conclusions: Simulation-based bronchoscopy training is effective in comparison with no intervention. Comparative effectiveness studies are few.


Procedural Virtual Reality Simulation Training for Robotic Surgery: a Randomised Controlled Trial Raison N. et al., Surgical Endoscopy, 2021
Background: Virtual reality (VR) training is widely used for surgical training, supported by comprehensive, high-quality validation. Technological advances have enabled the development of procedural-based VR training. This study assesses the effectiveness of procedural VR compared to basic skills VR in minimally invasive surgery.

Methods: 26 novice participants were randomised to either procedural VR (n = 13) or basic VR simulation (n = 13). Both cohorts completed a structured training program. Simulator metric data were used to plot learning curves. All participants then performed parts of a robotic-assisted radical prostatectomy (RARP) on a fresh frozen cadaver. Performances were compared against a cohort of 9 control participants without any training experience. Performances were video recorded and assessed blindly using GEARS post hoc.

Results: Learning curve analysis demonstrated improvements in technical skill for both training modalities although procedural training was associated with greater training effects. Any VR training resulted in significantly higher GEARS scores than no training (GEARS score 11.3 ± 0.58 vs. 8.8 ± 2.9, p = 0.002). Procedural VR training was found to be more effective than both basic VR training and no training (GEARS 11.9 ± 2.9 vs. 10.7 ± 2.8 vs. 8.8 ± 1.4, respectively, p = 0.03).

Conclusions: This trial has shown that a structured programme of procedural VR simulation is effective for robotic training with technical skills successfully transferred to a clinical task in cadavers. Further work to evaluate the role of procedural-based VR for more advanced surgical skills training is required.


Arthroscopic Training Courses Improve Trainee Arthroscopy Skills: A Simulation-Based Prospective Trial Martin KD et al, Arthroscopy, March 2016

Purpose: To evaluate the correlation between timed task performance on an arthroscopy shoulder simulator and participation in a standardized expert shoulder arthroscopy educational course.

Methods: Orthopaedic trainees were voluntarily recruited from over 25 residency programs throughout the United States and Canada. Each trainee was tested on arrival at the Arthroscopy Association of North America orthopaedic learning center on a virtual reality arthroscopy shoulder simulator, and his or her performance was objectively scored. Each trainee’s postgraduate year level was recorded, as was his or her experience in residency with shoulder arthroscopy as measured by Accreditation Council for Graduate Medical Education case-log totals. After the focused 4-day training curriculum consisting of didactics and cadaveric experience, each trainee was re-evaluated on the same simulator. Statistical analysis was performed to determine if participation in the course was associated with changes in simulation performance from before to after assessment.

Results: Forty-eight trainees completed the testing. On completion of the course, trainees showed significant improvements in all objective measures recorded by the simulator. Total probe distance needed to complete the task decreased by 42% (from 420.4 mm to 245.3 mm, P < .001), arthroscope tip distance traveled decreased by 59% (from 194.1 mm to 80.2 mm, P < .001), and time to completion decreased by 38% (from 66.8 seconds to 41.6 seconds, P < .001). Highly significant improvements in all 3 measures suggest improved instrument handling, anatomic recognition, and arthroscopy-related visual-spatial ability.

Conclusions: This study shows objective improvement in orthopaedic trainee basic arthroscopy skill and proficiency after a standardized 4-day arthroscopy training curriculum. The results validate the Arthroscopy Association of North America resident training course and its curriculum with objective evidence of benefit.

Level of evidence: Level III, prospective study of nonconsecutive participants.


Standardization of Bronchoscopy Training Across Yorkshire and Humber Deanery F Chowdhury, S Pathmanathan, I Aslam, A Arnold, J Hogg, S Renshaw, J Hill, N Chaudhuri, T Rogers, J Kastelik. The abstract was presented at the British Thoracic Society June 13-14, 2013, University Place Conference Centre, University of Manchester, UK

Background Reduced exposure to bronchoscopyis a key issue for respiratory trainees with effect on their confidence in undertaking the procedure and thus patient safety. Studies have shown that simulation improves confidence in bronchoscopy skills but do not explore the most optimal teaching methods.

Aim To assess two different methods of delivering bronchoscopy simulation training

Methods Two half day simulation bronchoscopy courses were designed independently within the Yorkshire and Humber Deanery. Course 1 concentrated on providing a knowledge based training consisting of a didactic lecture followed by equal time spent on a Symbionixsimulator and on the BTS e-learning hub website. Course 2 provided pre-course material in the form of BTS guidelines and bronchoscopy procedure pocketbook. The course focused on hands-on simulation training using a bronchoscopy manikin and the Symbionix simulator. All candidates completed pre and post course Likert scale questionnaires in six areas relating to participant knowledge and confidence in using a bronchoscope.

Results Overall 30 trainees; 15 in each course were evaluated. Candidates had performed between 0 to >300 previous bronchoscopies and were from across the SpR years. Both courses delivered significant improvement in confidence scores in all of the six areas assessed. The greatest improvement was found in confidence levels in technical ability . Course 1 candidates showed a greater confidence improvement in factual skills (such as knowledge of contra-indications of the procedure and anatomy). Course 2 demonstrated that 93% of candidates agreed that the simulator helped to improve technical ability in contrast to 100% with manikin exposure. 100% of candidates found the pocketbook was a useful adjuvant to the course with 93% agreeing that they would find this useful to complement their training.


Arthroscopy Skills Development with a Surgical Simulator Rebolledo BJ et al, The American Journal of Sports Medicine, May 29, 2015

Background:  Surgical simulation has become increasingly relevant to orthopaedic surgery education and could translate to improved operating room proficiency in orthopaedic surgery trainees.

Purpose: To compare the arthroscopic performance of junior orthopaedic surgery residents who received training with a knee and shoulder arthroscopy surgical simulator with those who received didactic training.
Study Design: Controlled laboratory study.

Methods: Fourteen junior orthopaedic surgery residents at a single institution were randomized to receive knee and shoulder arthroscopy training with a surgical simulator (n = 8) or didactic lectures with arthroscopy models (n = 6). After their respective training, performance in diagnostic knee and shoulder arthroscopy was assessed using a cadaveric model. Time to completion and assessment of arthroscopic handling using a subjective injury grading index (scale, 1-10) was then used to evaluate performance
in final cadaveric testing.

Results: Orthopaedic surgery residents who trained with a surgical simulator outperformed the didactic-trained residents in shoulder arthroscopy by time to completion (–35%; P = .02) and injury grading index (–35%; P = .01). In addition, a trend toward improved performance of knee arthroscopy by the simulator-trained group was found by time to completion (–36%; P = .09) and injury grading index (P = .08).

Conclusion: In this study, junior orthopaedic surgery residents who trained with a surgical simulator demonstrated improved arthroscopic performance in both knee and shoulder arthroscopy. However, future validation of surgical simulator training for orthopaedic surgery residents remains warranted.

Clinical Relevance: Surgical skill development with an arthroscopy surgical simulator could translate to improved arthroscopy performance in the operating room.


Evaluation Of Procedural Virtual Reality Simulation Training: A Randomised Controlled Trial Raison N. et al, The Journal of Urology, 2019

Introduction and Objectives: Improvements in virtual reality (VR) technology have enabled the development of procedural simulation training, which closely replicate surgical procedures. VR simulation training has been shown to be highly effective for robotic surgical training, however to-date curricula are limited to generic basic skills training. This RCT aims to compare the transfer of learning following procedural VR or standard basic skills VR training.

Methods: Initially 25 novice surgeons underwent basic robotic skills training, completing three FRS tasks. Participants were then block randomised to standard basic VR training or procedural VR training. All training was performed on the RobotiX Mentor (3D Systems (Airport City, Israel) VR robotic simulator. Standard basic skills training comprised further training following the FRS curriculum. The procedural simulation group underwent training on the guided bladder neck dissection and guided urethrovesical anastomosis tasks, parts of radical prostatectomy training module. Both groups completed a total of at least 5 hours of training. Following training both groups underwent transfer of skills assessment on fresh frozen cadavers using a Da Vinci Xi surgical robot in a simulated operating room environment. Their performances were compared to a control group of novice training without training. All performances were video recorded and were assessed blindly post hoc by a trained expert using GEARS.

Results: Baseline FRS scores were equal between the two groups (p=0.5). Subjects in both arms completed an average of 5.6 ± 0.3 hours of training. VR training (basic or procedural) resulted in a significantly higher GEARS score than no training, (mean GEARS score 11.3 ± 0.6 vs 8.8 ± 2.9 p=0.002). Procedural training resulted in significantly higher GEARS score than either basic training or control (p=0.03)(Figure 1).

Conclusions: This study provides the first evidence supporting the use of procedural-based VR simulation for training robotic skills even in novice participants. It also provides further validity evidence to support the use of VR simulation and the effective transfer of learned skills.


The Validation of a Novel Robot-Assisted Radical Prostatectomy Virtual Reality Module Harrison et al., Journal of Surgical Education, 2018

Objective: To perform the first validation of a full procedural virtual reality robotic training module and analysis of novice surgeon’s learning curves.

Design: Participants completed the bladder neck dissection task and urethrovesical anastomosis task (UVA) as part of the prostatectomy module. Surgeons completed feedback questionnaires assessing the realism, content, acceptability and feasibility of the module. Novice surgeons completed a 5.5-hour training programme using both tasks.

Setting: King’s College London, London

Participants: 13 novice, 24 intermediate and 8 expert surgeons completed the validation study.

Results: Realism was scored highly for BDN (mean 3.4/5) and UVA (3.74/5), as was importance of BDN (4.32/5) and UVA (4.6/5) for training. It was rated as a feasible (3.95/5) and acceptable (4/5) tool for training. Experts performed significantly better than novice group in 6 metrics in the UVA including time (p = 0.0005), distance by camera (p = 0.0010) and instrument collisions (p = 0.0033), as well as task-specific metrics such as number of unnecessary needle piercing points (p = 0.0463). In novice surgeons, a significant improvement in performance after training was seen in many metrics for both tasks. For bladder neck dissection task, this included time (p < 0.0001), instrument collisions (p = 0.0013) and total time instruments are out of view (p = 0.0251). For UVA, this included time (p = 0.0135), instrument collisions (p = 0.0066) and task-specific metrics such as injury to the urethra (p = 0.0032) and bladder (p = 0.0189).

Conclusions: Surgeons found this full procedural VR training module to be a realistic, feasible and acceptable component for a robotic surgical training programme. Construct validity was proven between expert and novice surgeons. Novice surgeons have shown a significant learning curve over 5.5 hours of training, suggesting this module could be used in a surgical curriculum for acquisition of technical skills. Further implementation of this module into the curriculum and continued analysis would be beneficial to gauge how it can be fully utilised.


Curriculum-based solo virtual reality training for laparoscopic intracorporeal knot tying: objective assessment of the transfer of skill from virtual reality to reality Munz Y. et al, Am J Surg . 2007

Background: Very few studies have addressed the transferability of skills from virtual reality (VR) to real life. The aim of this study was to assess the feasibility and effectiveness of teaching intracorporeal knot tying (ICKT) by VR simulation only.

Methods: Twenty novices underwent structured training of basic skills training on the Minimally Invasive Surgical Trainer simulator (Mentice AB, Gothenburg, Sweden) followed by knot tying training on the LapSim simulator (Surgical Science, Gothenburg, Sweden). They were assessed pre- and post-training on a video trainer. Assessment of performance included motion tracking and video-based checklist. Nonparametric statistical analysis was used, and P < .05 was deemed significant.

Results: All participants completed a correct knot as compared with only 25% before VR training. Time to completion was 66% faster and knot quality 45% better after VR training. Significant reduction in number of movements (P = .006) and distance traveled (P < .000) by both hands after VR training.

Conclusions: Teaching ICKT by VR simulators only is feasible and effective. Furthermore, this study highlights the complementary use of different VR simulators within a structured curriculum.


Skip to navigation