400+ Validation Studies

Objective
To assess whether real-time cystourethroscopy (CUS) performance improves by simulator-based training (criterion or predictive validity), addressing the research question ‘Does practical skills training on the URO Mentor (UM, Simbionix USA Corp., Cleveland, OH, USA) virtual-reality simulator improve the performance of flexible CUS in patients’.

Subjects and Methods
Participants (71 interns from Catharina Hospital Eindhoven, CHE, and 29 from University Medical Centre Groningen, UMCG) were randomized to carry out CUS in a patient after training on the UM (UM trained, 50) or without training on UM (control, 50). The assessment of real-time performance consisted of scoring on a Global Rating Scale (GRS) by supervisors unaware of training status. Data were analysed using stepwise  multiple linear regression. The effect size (ES) indication for correlations was used to interpret the magnitude of a standard regression coefficient (β); an ES of 0.10, 0.30 and 0.50 were considered small, moderate and large, respectively. The study was approved by the Medical Review Ethics Committees of the participating hospitals.

Results
Overall, the group that received training performed significantly better than the controls (P ≤ 0.003, β range 0.30–0.47). There was no effect of training for participants with a specific preference for a surgical  speciality in two of five GRS scores. Participants from CHE obtained higher GRS 3 scores than those from UMCG. Significantly more UMCG trainees indicated having had stress than those from CHE (P < 0.001).

Conclusions
The results showed that interns who had trained on UM outperformed controls for a CUS procedure in a patient. Training for CUS on the UM is to be recommended for learning to respect tissue, procedural knowledge, flow of procedure and forward planning. Use of the UM to train interns with a specific interest in a surgical speciality in handling instruments, and time and motion, seems to be of limited value.

Keywords
randomized controlled trial, simulation, training, transfer, cysto-urethroscopy, validation

Abstract:
Objective Diferent simulation modalities may be utilised in a curricular fashion to beneft from the strengths of each training model. The aim of this study is to evaluate a novel multi-modality ureterorenoscopy (URS) simulation curriculum in terms of educational value, content validity, transfer of skills and inter-rater reliability.

Methods:
This international prospective study recruited urology residents (n=46) with≤10 URS experience and no prior simulation training. Participants were guided through each phase of the expert-developed SIMULATE URS curriculum by trainers and followed-up in the operating room (OR). Video recordings were obtained during training. A post-training evaluation survey was distributed to evaluate content validity and educational value, using descriptive statistics. Performance was evaluated using the objective structured assessment of technical skills (OSATS) scale to measure improvement in scores throughout the curriculum. Pearson’s correlation coefcient and Cohen’s kappa tests were utilised to investigate correlation and agreement between raters.

Results:
Participants reported gaining OR-transferrable skills (Mean: 4.33±0.67) and demonstrated marked improvement in throughout the curriculum, transferred to the OR for both semi-rigid URS (p=0.004) and fexible URS (p=0.007). 70% of participants were successfully followed-up in the OR (n=32). No diferences were identifed with the additional use of fresh frozen cadavers (p=0.85, p=0.90) and the URO Mentor VR simulator (p=0.13, p=0.22). A moderate level of correlation was noted on the video OSATS assessments, between two expert assessors (r=0.70), but a poor agreement with the live rating.

Conclusion:
The SIMULATE URS training curriculum received high educational value from participants, who demonstrated statistically signifcant improvement with consecutive cases throughout the curriculum and  transferability of skills to the OR in both semi-rigid and fexible URS.

Keywords Education · Simulation · Urology training · Ureterorenoscopy · Curriculum

Results: Twenty-four physicians completed the study. The mean improvement in EVARATE score during the course was +11.8 (95% confidence interval 9.8 – 13.7) points (p < .001). Twenty-two participants (92%) passed with a mean number of 2.8 ± 0.7 test attempts to reach the pass limit. Cronbach’s alpha coefficient was 0.91, corresponding to excellent reliability of the EVARATE scale. Differences between instructors’ EVARATE ratings were insignificant (p = .16), with a maximum variation between instructors of ± 1.3 points.

Background

Point-of-care ultrasound is rapidly gaining traction in clinical practice, including primary care. Yet, logistical challenges and geographical isolation hinder skill acquisition. Concurrently, an evidentiary gap exists concerning such guidance’s effectiveness and optimal implementation in these settings.

Methods

We developed a lung point-of-care ultrasound (POCUS) curriculum for primary care physicians in a rural, medically underserved region of the south of Israel. The course included recorded lectures, pre-course assessments, hands-on training, post-workshop lectures, and individual practice. To evaluate our course, we measured learning outcomes and physicians’ proficiency in different lung POCUS domains using hands-on technique assessment and gathered feedback on the course with a multi-modal perception approach: an original written pre- and post-perception and usage questionnaire.

Results

Fifty primary care physicians (PCPs) showed significant improvement in hands-on skills, increasing from 6 to 76% proficiency (p < 0.001), and in identifying normal versus abnormal views, improving from 54 to 74% accuracy (p < 0.001). Ten weeks after training, primary care physicians reported greater comfort using lung ultrasound, rising from 10 to 54% (p < 0.001), and improved grasp of its potential and limits, increasing from 27.5% to 84% (p < 0.001). Weekly usage increased from none to 50%, and the number of primary care physicians not using at all decreased from 72 to 26% (p < 0.001).

Conclusions

A two-day focused in-person and remote self-learning lung-POCUS training significantly improved primary care physicians’ lung ultrasound skills, comfort, and implementation.

Objectives: Virtual training simulators have been introduced in several surgical disciplines to improve residents’ abilities. Through the use of the LapSim® virtual training simulator (Surgical Science, Göteborg, Sweden), this study aims to plan an effective learning path in minimally invasive thoracic and general surgery.

Methods: All thoracic and general surgery trainees in their 1st and 2nd year of residency at the University of Insubria were enrolled and randomized into 2 groups: residents undergoing an intensive twice-a-week virtual training programme (group A: n = 8) and those undergoing a once-weekly non-intensive virtual training programme (group B: n = 9). The virtual training programme was divided into 4 modules, each of 12 weeks. In the 1st module, trainees repeated grasping, cutting, clip application, lifting and grasping, and fine dissection exercises during each training session. Seal-and-cut exercise was performed as the initial and final test. Data on surgical manoeuvres (time and on mistakes) were collected; intra- and inter-group comparisons were planned.

Results: No significant differences were observed between groups A and B at the 1st session, confirming that the 2 groups had similar skills at the beginning. After 12 weeks, both groups showed improvements, but comparing data between initial and final test, only Group A registered a significant reduction in total time (P-value = 0.0015), left (P-value = 0.0017) and right (P-value = 0.0186) instrument path lengths, and in left (P-value = 0.0010) and right (P-value = 0.0073) instrument angular path lengths, demonstrating that group A acquired greater precision in surgical manoeuvres.

Conclusions: Virtual simulator training programme performed at least twice a week was effective for implementing basic surgical skills required for the trainee’s professional growth. Additional virtual training modules focused on more complex exercises are planned to confirm these preliminary results.

Background

This pilot study aims to examine the effectiveness of a spinal cord stimulator (SCS) simulator training system in improving the confidence of pain fellows in SCS placement.

Methodology

Five Ukrainian physicians (neurologists, neurosurgeons, and an anesthesiologist) completed a 10-item survey regarding their confidence in various aspects of SCS placement and their opinions on how effective SCS models were for educational purposes. After placing SCS leads using the SCS simulator, the physicians took the same survey again. The Mann-Whitney U test was used to determine if there was a significant difference in total scores pre and post-simulator training. The software PAST (PAleontological STatistics) was used for statistical analysis.

Results

Overall, five participants had a 38% statistically significant increase in survey scores before and after the intervention (mean: 4.2 vs. 6.2, p = 0.0055). With regards to each item of the survey, participants had a significantly increased confidence in driving leads (2.6 vs. 5.2, p = 0.008) and in overall technical skills for the SCS procedure after the training (2.8 vs. 5.2, p = 0.0188). Although the other eight survey items were not statistically significant (p > 0.05), participants had a 28% increase in confidence when inserting epidural needles, a 20% increase in interpreting simulated X-rays, a 32% increase in navigating challenging anatomical variations, a 12% increase in identifying key anatomical landmarks, a 20% increase in ensuring the correct placement of the lead, or a 53% increase in preparedness for performing an SCS procedure in a real clinical setting. The participant’s perspective on how valuable the stimulator training was for enhancing procedural skills increased by 38% and how well the simulator replicated real-life SCS procedure increased by 52%, although both were statistically insignificant (p > 0.05).

Conclusions

This pilot study shows that the utilization of simulated neuromodulation training is a viable means of augmenting neuromodulation education by increasing physician’s confidence in aspects of the SCS placement procedure. The extent to which simulator training improves procedural skills in a real-life SCS placement needs to be investigated further.

Background: There is a lack of evidence regarding the effectiveness of virtual simulators as a means to acquire hands-on exposure to endoscopic retrograde cholangiopancreatography (ERCP). The present study aimed to assess the outcome and construct validity of virtual ERCP when training on the GI II Mentor simulator.

Methods: A group of seven experienced endoscopists were compared with 31 novices. After a short introduction, they were requested to carry out three virtual ERCP procedures: diagnosing and removing a common bile duct (CBD) stone; diagnosing and taking brush cytology from a hilar stenosis; and, finally, diagnosing and treating a cystic leakage with a BD stent. For each task, the total time required to complete the task, time required to correctly view the papilla, total time of irradiation, time to deep cannulation, time to define diagnosis, time to complete sphincterotomy, and time to complete the respective intervention were measured. Cannulation of the BD, correct diagnosis, sphincterotomy, and time to complete intervention were assessed by an assessor blinded to the status of the endoscopist who performed the virtual ERCP.

Results: The time required to visualize the papilla and to cannulate deeply when removing the BD stone was significantly shorter for the experts (both p < 0.05). The time to visualize the papilla, cannulate deeply, reach a diagnosis, complete sphincterotomy, and complete the intervention was significantly shorter for the experts when managing cystic leakage (all p < 0.05). In diagnosing and taking brush cytology from a hilar stenosis, there was only a trend toward the experts needing less time for the deep cannulation of the BD (p = 0.077).

Conclusion: The performance differed between experts and novices, especially in the management of cystic leakage. This corroborates the construct validity of the GI II Mentor simulator.

Background The Fundamentals of Endoscopic Surgery™ (FES) program consists of online materials and didactic and skills-based tests. All components were designed to measure the skills and knowledge required to perform safe flexible endoscopy. The purpose of this multicenter study was to evaluate the reliability and validity of the hands-on component of the FES examination, and to establish the pass score.

Methods Expert endoscopists identified the critical skill set required for flexible endoscopy. They were then modeled in a virtual reality simulator (GI Mentor™ II, Simbionix™ Ltd., Airport City, Israel) to create five tasks and metrics. Scores were designed to measure both speed and precision. Validity evidence was assessed by correlating performance with self-reported endoscopic experience (surgeons and gastroenterologists [GIs]). Internal consistency of each test task was assessed using Cronbach’s alpha. Test–retest reliability was determined by having the same participant perform the test a second time and comparing their scores. Passing scores were determined by a contrasting groups methodology and use of receiver operating characteristic curves.

Results A total of 160 participants (17 % GIs) performed the simulator test. Scores on the five tasks showed good internal consistency reliability and all had significant correlations with endoscopic experience. Total FES scores correlated 0.73, with participants’ level of endoscopic experience providing evidence of their validity, and their internal consistency reliability (Cronbach’s alpha) was 0.82. Test–retest reliability was assessed in 11 participants, and the intraclass correlation was 0.85. The passing score was determined and is estimated to have a sensitivity (true positive rate) of 0.81 and a 1-specificity (false positive rate) of 0.21.

Conclusions The FES hands-on skills test examines the basic procedural components required to perform safe flexible endoscopy. It meets rigorous standards of reliability and validity required for high-stakes examinations, and, together with the knowledge component, may help contribute to the definition and determination of competence in endoscopy.

Keywords: GAGES; Patient Care; Practice-Based Learning and Improvement; Systems-Based Practice; flexible endoscopy; simulation; skills assessment.

Background Fundamentals of Endoscopic Surgery (FES) has become a board certification requirement for general surgery residents. While the FES program provides a robust didactic curriculum, an endoscopic skills curriculum is lacking for this high stakes assessment. The aims of this study were to develop a proficiency-based endoscopic skills curriculum and assess its effectiveness on success in the FES exam.

Methods Endoscopy experts developed a multiple-choice questionnaire based on the FES online didactics. Five training cases from the GI Mentor II simulator were selected, and expert performance defined proficiency levels for each case. Participating surgery residents were required to review online didactics and achieve proficiency twice on selected simulator cases. The multiple-choice questionnaire, simulator-generated metrics of two endoscopy cases, Global Assessment of Gastrointestinal Endoscopic Skills (GAGES), NASA-Task Load Index (TLX), and the manual portion of the FES exam were used for assessment before and after training. The curriculum was implemented either alongside a clinical endoscopy rotation or independent of the rotation. Clinical endoscopic skills of participants with a dedicated rotation were assessed using GAGES.

Results Twenty-eight general surgery residents (PGY 2–5) participated in the study, of which 25 (89%) completed the curriculum. Scores of the multiple-choice questionnaire and all simulator-generated metrics improved in the post-training assessment, with the exception of the percentage of mucosal surface examined, which was slightly decreased. Simulated and clinical GAGES scores and the NASA-TLX score improved after training. Performance scores on four of five FES exam tasks were significantly improved.

Conclusions The proficiency-based endoscopic skills curriculum was successfully implemented both alongside the clinical endoscopy rotation and independent of the rotation. Participating residents acquired skills to pass the FES exam. This curriculum will be valuable to general surgery residency programs

Endobronchial ultrasound-guided transbronchial needle aspiration (EBUS-TBNA) is very operator dependent and has a long learning curve. Simulation-based training might shorten the learning curve, and an assessment tool with solid validity evidence could ensure basic competency before unsupervised performance.

A total of 16 respiratory physicians, without EBUS experience, were randomised to either virtual-reality simulator training or traditional apprenticeship training on patients, and then each physician performed EBUS-TBNA procedures on three patients. Three blinded, independent assessor assessed the video recordings of the procedures using a newly developed EBUS assessment tool (EBUSAT).

The internal consistency was high (Cronbach’s α=0.95); the generalisability coefficient was good (0.86), and the tool had discriminatory ability (p<0.001). Procedures performed by simulator-trained novices were rated higher than procedures performed by apprenticeship-trained novices: mean±sd are 24.2±7.9 points and 20.2±9.4 points, respectively; p=0.006. A pass/fail standard of 28.9 points was established using the contrasting groups method, resulting in 16 (67%) and 20 (83%) procedures performed by simulator-trained novices and apprenticeship-trained novices failing the test, respectively; p<0.001.

The endobronchial ultrasound assessment tool could be used to provide reliable and valid assessment of competence in EBUS-TBNA, and act as an aid in certification. Virtual-reality simulator training was shown to be more effective than traditional apprenticeship training.

Background: Virtual reality (VR) simulators are increasingly being used for surgical skills training. It is unclear what skills are best improved via VR, translate to live surgical skills, and influence patient outcomes.

Objective: To assess surgeons in VR and live surgery using a suturing assessment tool and evaluate the association between technical skills and a clinical outcome.

Design, setting, and participants: This prospective five-center study enrolled participants who completed VR suturing exercises and provided live surgical video. Graders provided skill assessments using the validated End-To-End Assessment of Suturing Expertise (EASE) suturing evaluation tool.

Outcome measurements and statistical analysis: A hierarchical Poisson model was used to compare skill scores among cohorts and evaluate the association of scores with clinical outcomes. Spearman’s method was used to assess correlation between VR and live skills.

Results and limitations: Ten novices, ten surgeons with intermediate expertise (median 64 cases, interquartile range [IQR] 6-80), and 26 expert surgeons (median 850 cases, IQR 375-3000) participated in this study. Intermediate and expert surgeons were significantly more likely to have ideal scores in comparison to novices for the subskills needle hold angle, wrist rotation, and wrist rotation needle withdrawal (p < 0.01). For both intermediate and expert surgeons, there was positive correlation between VR and live skills for needle hold angle (p < 0.05). For expert surgeons, there was a positive association between ideal scores for VR needle hold angle and driving smoothness subskills and 3-mo continence recovery (p < 0.05). Limitations include the size of the intermediate surgeon sample and clinical data limited to expert surgeons.

Conclusions: EASE can be used in VR to identify skills to improve for trainee surgeons. Technical skills that influence postoperative outcomes may be assessable in VR.

Background: Although simulation-based bronchoscopy has been shown to be an effective training modality, formal assessment should still be performed as new technology emerges. We sought to validate a simulator in essential bronchoscopic tasks, and survey perceptions of bronchoscopists on simulation.

Methods: A cohort study at 2 medical centers used 3 groups to assess construct validity of the Simbionix Bronchoscopy Simulator: 7 first-year fellows with <10 bronchoscopies each (novice), 6 pulmonologists with 200 to 1000 bronchoscopies each (experienced), and 7 pulmonologists with >1000 bronchoscopies each (expert). Participants were tested in 4 tasks (1: scope manipulation, 2: guided anatomic navigation, 3: airway anatomy, and 4: lymph node anatomy). Participants were scored and surveyed on their impressions of simulation. The means and Kruskal-Wallis test among groups were compared by task item (P<0.05).

Results: There were statistically significant differences in mean ranks among groups for tasks 1 and 3. For task 1, final score, time, mid-lumen time, and wall hits were discriminative (P=0.006, 0.006, 0.012, and 0.014, respectively). For task 3, time, bronchial segments identified, bronchial segments incorrectly identified, and bronchial segments skipped were discriminative (P=0.04, 0.012, 0.013, and 0.013, respectively). There was no statistically significant difference for task 2 and task 4. All participants agreed that simulation training is helpful and should be incorporated into bronchoscopic training.

Conclusions: The simulator demonstrated validity in differentiating skill in scope manipulation and airway anatomy, but did not discriminate skill levels in anatomic orientation or identification of lymph nodes. Bronchoscopy simulation was viewed as helpful by all levels and should be considered before performance on patients.

Background: Basic thoracic ultrasound is being used more frequently by clinicians in several settings due to its high diagnostic accuracy for many common causes of respiratory failure and dyspnoea. However, ultrasound examinations are operator-dependent, and sufficient competences are needed to obtain high sensitivity and specificity of examinations. Additionally, it is crucial for ultrasound operators to perceive the competence to interpret the images and integrate them into the patient history and other examinations. This study aims to explore and gather validity evidence for an objective structured clinical examination test of basic thoracic ultrasound competences and establish a pass/fail score.

Methods An expert panel created the test which included two theoretical and five practical stations representing cases with different diagnoses that cause respiratory symptoms and which are possible to establish by basic thoracic ultrasound. Twenty-five participants with different levels of experience in basic thoracic ultrasound completed the test. Data of the test scores were used for item analysis, and exploring validity evidence was done according to Messick’s framework which is recommended. The contrasting groups’ standard-setting method was used to establish a pass/fail score.

Results The summarised internal consistency reliability was high with a Cronbach’s alpha of 0.87. The novice group (n = 4) had a mean test score of 42 ± 10.1 points, the intermediate group (n = 8) scored 79.1 ± 8.1 points, and the experienced group (n = 13) 89.0 ± 6.2 points (one-way ANOVA, p < 0.001). A pass/fail score of 71 points was thus derived (maximum test score = 105 points).

Conclusion We developed a test for the assessment of clinical competences in basic thoracic ultrasound with solid validity evidence, and a pass/fail standard with no false positives or false negatives.

Objective: Surgical techniques are learned gradually throughout an orthopedic residency. Training on real patients carries drawbacks such as limited access and elevated risk. Alternatively, surgical simulation allows residents to practice in a safe environment with greater access to standardized surgical tasks. Virtual reality simulators display images inside an artificial joint, often providing real-time haptic feedback to allow for realistic interaction. The objective of this study was to evaluate the construct validity of a virtual reality simulator for knee arthroscopy by analyzing the capacity of system parameters to distinguish between expert and novice surgeons.

Design: This comparative cross-sectional study contrasts the automated performance reports for novice and expert orthopedic surgeons after executing surgical tasks on the ARTHRO Mentor virtual reality simulator. Setting Surgical simulation center at the University of Chile Clinical Hospital, Santiago, Chile.

Participants: The novice group consisted of 20 second-year orthopedic and traumatology residents at the University of Chile School of Medicine. The expert group consisted of 10 experienced arthroscopic surgeons. All participants carried out standardized tasks in the knee arthroscopy virtual reality simulator. The median performance scores of the two groups were compared, and multivariate logistic regression was performed to assess the capacity of the system to discriminate between the two groups.

Results: Median performance on the vast majority of surgical tasks was superior for the expert group. The expert group had performance values equal to or higher than the novice group on 43 of the 44 variables recorded for the basic tasks and 74 of the 75 advanced task variables. The multivariate logistic regression analysis discriminated expert from novice users with 100% accuracy.

Conclusion: The virtual reality simulator for knee arthroscopy showed good construct validity, with performance metrics accurately discriminating between expert and novice users.

Objective:  To assess the effectiveness of simulation-based teaching vs traditional teaching of TEE knowledge and skills of cardiology fellows.

Design, Setting, and Participants:  Between November 2020 and November 2021, all consecutive cardiology fellows inexperienced in TEE from 42 French university centers were randomized (1:1; n = 324) into 2 groups with or without simulation support.

Main Outcomes and Measures:  The co-primary outcomes were the scores in the final theoretical and practical tests 3 months after the training. TEE duration and the fellows’ self-assessment of their proficiency were also assessed.

Results:  While the theoretical and practical test scores were similar between the 2 groups (324 participants; 62.6% male; mean age, 26.4 years) before the training (33.0 [SD, 16.3] points vs 32.5 [SD, 18.5] points; P = .80 and 44.2 [SD, 25.5] points vs 46.1 [SD, 26.1] points; P = .51, respectively), the fellows in the simulation group (n = 162; 50%) displayed higher theoretical test and practical test scores after the training than those in the traditional group (n = 162; 50%) (47.2% [SD, 15.6%] vs 38.3% [SD, 19.8%]; P < .001 and 74.5% [SD, 17.7%] vs 59.0% [SD, 25.1%]; P < .001, respectively). Subgroup analyses showed that the effectiveness of the simulation training was even greater when performed at the beginning of the fellowship (ie, 2 years or less of training) (theoretical test: an increase of 11.9 points; 95% CI, 7.2-16.7 vs an increase of 4.25 points; 95% CI, −1.05 to 9.5; P = .03; practical test: an increase of 24.9 points; 95% CI, 18.5-31.0 vs an increase of 10.1 points; 95% CI, 3.9-16.0; P < .001). After the training, the duration to perform a complete TEE was significantly lower in the simulation group than in the traditional group ( 8.3 [SD, 1.4] minutes vs 9.4 [SD, 1.2] minutes; P < .001, respectively). Additionally, fellows in the simulation group felt more ready and more confident about performing a TEE alone after the training (mean score, 3.0; 95% CI, 2.9-3.2 vs mean score, 1.7; 95% CI, 1.4-1.9; P < .001 and mean score, 3.3; 95% CI, 3.1-3.5 vs mean score, 2.4; 95% CI, 2.1-2.6; P < .001, respectively).

Conclusions and Relevance:  Simulation-based teaching of TEE showed a significant improvement in the knowledge, skills, and self-assessment of proficiency of cardiology fellows, as well as a reduction in the amount of time needed to complete the examination. These results should encourage further investigation of clinical performance and patient benefits of TEE simulation training.

Andrea Moglia, Luca Morelli, Vincenzo Ferrari, Mauro Ferrari, Franco Mosca, Alfred Cuschieri

ABSTRACT

Background: There is an increasing interest for a test assessing objectively the innate aptitude for surgery as a craft specialty to complement the current selection process of surgical residents. The aim of this study was to quantify the size of individuals with high, average, and low level of innate psychomotor skills among medical students.

Methods: A volunteer sample of 155 medical students, without prior experience with surgical simulator, executed five tasks at a virtual simulator for robot-assisted surgery. They had to reach proficiency twice consecutively in each before moving to the next one. A weighting based on time and number of attempts needed to reach proficiency was assigned to each task.

Results: Nine students (5.8%) out of 155 significantly outperformed all the others on median (i.q.r.) weighted time [44.7 (42.2–47.3) min vs. 98.5 (70.8–131.8) min, p < 0.001], and number of attempts to reach proficiency [14 (12–15) vs. 23 (19–32.75), p < 0.001). Seventeen students (11.0%) scored significantly much worse than the rest on median weighted time [202.2 (182.5–221.0) min vs. 84.3 (65.7–114.4) min, p < 0.001], and number of attempts [42 (40–48) vs. 22 (17.25–28), p < 0.001]. Low correlation between simulator scores and extracurricular activities, like videogames and musical instruments, was found.

Conclusions: The test successfully identified two groups straddling the large cohort with average innate aptitude for psychomotor skills: (i) innately gifted and (ii) with scarce level. Hence, exercises on a virtual simulator are a valid test of innate manual dexterity and can be considered to complement the selection process for a surgical training program, primarily to identify individuals with low innate aptitude for surgery and advise them to consider specialization in other (non-craft) medical specialties.

Background and Purpose: Surgery is a high-stakes “performance.” Yet, unlike athletes or musicians, surgeons do not engage in routine “warm-up” exercises before “performing” in the operating room. We study the impact of a preoperative warm-up exercise routine (POWER) on surgeon performance during laparoscopic surgery.

Materials and Methods: Serving as their own controls, each subject performed two pairs of laparoscopic cases, each pair consisting of one case with POWER (+POWER) and one without (–POWER). Subjects were randomly assigned to +POWER or −POWER for the initial case of each pairing, and all cases were performed ≥1 week apart. POWER consisted of completing an electrocautery skill task on a virtual reality simulator and 15 minutes of laparoscopic suturing and knot tying in a pelvic box trainer. For each case, cognitive, psychomotor, and technical performance data were collected during two different tasks: mobilization of the colon (MC) and intracorporeal suturing and knot tying (iSKT). Statistical analysis was performed using SYSTAT v11.0.

Results: A total of 28 study cases (14+POWER, 14−POWER) were performed by seven different subjects. Cognitive and psychomotor performance (attention, distraction, workload, spatial reasoning, movement smoothness, posture stability) were found to be significantly better in the +POWER group (P≤0.05) and technical performance, as scored by two blinded laparoscopic experts, was found to be better in the +POWER group for MC (P=0.04) but not iSKT (P=0.92). Technical scores demonstrated excellent reliability using our assessment tool (Cronbach ∝=0.88). Subject performance during POWER was also found to correlate with intraoperative performance scores.

Conclusions: Urologic trainees who perform a POWER approximately 1 hour before laparoscopic renal surgery demonstrate improved cognitive, psychomotor, and technical performance.

Background: Laparoscopic appendicectomy (LA) is a common surgical emergency procedure mainly performed by trainees. The aim was to develop a step-wise structured virtual reality (VR) curriculum for LA to allow junior surgeons to hone their skills in a safe and controlled environment.

Methods: A prospective randomized study was designed using a high-fidelity VR simulator. Thirty-five novices and 25 experts participated in the assessment and their performances were compared to assess construct validity. Learning curve analysis was performed.

Results: Five of the psychomotor tasks and all appendicectomy tasks showed construct validity. Learning was demonstrated in the majority of construct-valid tasks. A novel goal-directed VR curriculum for LA was constructed. Conclusions: A step-wise structured VR curriculum for LA is proposed with a framework which includes computer generated metrics and supports deliberate practice, spacing intervals, human instruction/feedback and assessment. Future study should test the feasibility of its implementation and transferability of acquired skill.

Background:   Robotic surgery simulation may provide a way for surgeons to acquire specific robotic surgical skills without practicing on live patients.

Methods:   Five robotic surgery experts performed 10 simulator skills to the best of their ability, and thus, established expert benchmarks for all parameters of these skills. A group of credentialed gynecologicsurgeons naive to robotics practiced the simulator skills until theywere able to perform each one as well as our experts. Within a weekof doing so, they completed robotic pig laboratory training, after whichthey performed supracervical hysterectomies as their first-ever livehuman robotic surgery. Time, blood loss, and blinded assessments ofsurgical skill were compared among the experts, novices, and a groupof control surgeons who had robotic privileges but no simulator expo-sure. Sample size estimates called for 11 robotic novices to achieve90% power to detect a 1 SD difference between operative times ofexperts and novices (> = 0.05).

Results:   Fourteen novice surgeons completed the studyVspendingan average of 20 hours (range, 9.7Y38.2 hours) in the simulation lab-oratory to pass the expert protocol. The mean operative times for the ex-pert and novices were 20.2 (2.3) and 21.7 (3.3) minutes, respectively(P = 0.12; 95% confidence interval, j1.7 to 4.7), whereas the mean timefor control surgeons was 30.9 (0.6) minutes (P G 0.0001; 95% confi-dence interval, 6.3Y12.3). Comparisons of estimated blood loss (EBL) andblinded video assessment of skill yielded similar differences betweengroups.

Conclusions:   Completing this protocol of robotic simulator skills translated to expert-level surgical times during live human surgery. As such, we have established predictive validity of this protocol.

Introduction: Recent advancements in surgical technology, reduced working hours, and training opportunities exacerbated by the COVID-19 pandemic have led to an increase in simulation-based training. Furthermore, a rise in endovascular procedures has led to a requirement for high-fidelity simulators that offer comprehensive feedback. This review aims to identify vascular surgery simulation models and assess their validity and levels of effectiveness (LoE) for each model in order to successfully implement them into current training curricula.

Methods: PubMed and EMBASE were searched on January 1, 2021, for full-text English studies on vascular surgery simulators. Eligible articles were given validity ratings based on Messick’s modern concept of validity alongside an LoE score according to McGaghie’s translational outcomes.

Results: Overall 76 eligible articles validated 34 vascular surgery simulators and training courses for open and endovascular procedures. High validity ratings were achieved across studies for: content , response processes , the internal structure , relations to other variables, and consequences. Only seven studies achieved an LoE greater than 3/5. Overall, ANGIO Mentor was the most highly validated and effective simulator and was the only simulator to achieve an LoE of 5/5.

Conclusions: Simulation-based training in vascular surgery is a continuously developing field with exciting future prospects, demonstrated by the vast number of models and training courses. To effectively integrate simulation models into current vascular surgery curricula and assessments, there is a need for studies to look at trainee skill retention over a longer period of time. A more detailed discussion on cost-effectiveness is also needed.

Objectives: Healthcare evolution requires optimisation of surgical training to provide safe patient care. Operating
room performance after completion of proficiency based training in vascular surgery has not been investigated.

Design: A randomised controlled trial evaluated the impact of a Proficiency based Stepwise Endovascular
Curricular Training program (PROSPECT) on the acquisition of endovascular skills and the transferability of these
skills to real life interventions.

Materials: All subjects performed two endovascular interventions treating patients with symptomatic iliac and/or
superficial femoral artery stenosis under supervision. Primary outcomes were technical performances (Global Rating
Scale [GRS]; Examiner Checklist), operative metrics, and patient outcomes, adjusted for case difficulty and trainee
experience. Secondary outcomes included knowledge and technical performance after 6 weeks and 3 months.

Methods: Thirty-two general surgical trainees were randomised into three groups. Besides traditional training, the first group (n ¼ 11) received e-learning and simulation training (PROSPECT), the second group (n ¼ 10) only had access to e-learning, while controls (n ¼ 11) did not receive supplementary training.

Results: Twenty-nine trainees (3 dropouts) performed 58 procedures. Trainees who completed PROSPECT showed
superior technical performance (GRS 39.36 +/- 2.05; Checklist 63.51 +/- 3.18) in real life with significantly fewer
supervisor takeovers compared with trainees receiving e-learning alone (GRS 28.42 +/- 2.15; p ¼ .001; Checklist
53.63 +/- 3.34; p ¼ .027) or traditional education (GRS 23.09 +/- 2.18; p ¼ .001; Checklist 38.72 +/- 3.38; p ¼ .001).
Supervisors felt more confident in allowing PROSPECT trained physicians to perform basic (p ¼ .006) and complex
(p ¼ .003) procedures. No differences were detected in procedural parameters (such as fluoroscopy time, DAP,
procedure time, etc.) or complications. Proficiency levels were maintained up to 3 months.

Conclusions: A structured, stepwise, proficiency-based endovascular curriculum including e-learning and
simulation-based training should be integrated early into training programs to enhance trainee performance.

Introduction: Simulator training can potentially provide high-intensity training in electrophysiology (EP) without compromising patient safety. We assessed the construct validity of a novel EP simulator (ANGIO Mentor; Simbionix) and developed proficiency-based scores for clinical EP simulator training.

Methods: Two European training centers participated. Participants were grouped on the basis of years of EP experience and (for a subset) subjectively scored clinical catheter manipulation skills. Each participant attempted the simulator’s 5 modules 3 times. These modules focus on catheter manipulation in 3-dimensional models, ranging from geometric shapes to fluoroscopic contracting cardiac models, with performance scored by the system on the basis of attainment of preset targets. Using these scores, targets were formulated for basic EP training.

Results: Twenty-eight participants were included (13 with subjectively scored catheter manipulation). Scores for participants with less than 1-year EP experience (group 1) were significantly lower for each of the attempts at the modules (P < 0.002). For group 1 only, scores improved with subsequent attempts (P < 0.005). In 4 of the 5 modules, scores of group 1 were significantly lower than the more experienced groups (P < 0.0005). Participants with subjectively scored above-average catheter manipulation skills also had higher scores in 4 of the 5 modules (P < 0.05). Target scores for a proficiency-based training program were generated from the median scores for each module for those with 1-year experience or more.

Conclusions: Scores attained in the simulator can distinguish those with less than 1-year EP experience and those with above-average catheter manipulation skills. Consequently, target scores have been generated for a proficiency-based training program.

Background:  Basic thoracic ultrasound is being used more frequently by clinicians in several settings due to its high diagnostic accuracy for many common causes of respiratory failure and dyspnoea. However, ultrasound examinations are operator-dependent, and sufficient competences are needed to obtain high sensitivity and specificity of
examinations. Additionally, it is crucial for ultrasound operators to perceive the competence to interpret the images and integrate them into the patient history and other examinations. This study aims to explore and gather validity evidence for an objective structured clinical examination test of basic thoracic ultrasound competences and establish
a pass/fail score.

Methods: An expert panel created the test which included two theoretical and five practical stations representing cases with different diagnoses that cause respiratory symptoms and which are possible to establish by basic thoracic ultrasound. Twenty-five participants with different levels of experience in basic thoracic ultrasound completed
the test. Data of the test scores were used for item analysis, and exploring validity evidence was done according to Messick’s framework which is recommended. The contrasting groups’ standard setting method was used to establish a pass/fail score.

Results:  The summarised internal consistency reliability was high with a Cronbach’s alpha of 0.87. The novice group (n = 4) had a mean test score of 42 ± 10.1 points, the intermediate group (n = 8) scored 79.1 ± 8.1 points, and the experienced group (n = 13) 89.0 ± 6.2 points (one-way ANOVA, p < 0.001). A pass/fail score of 71 points was thus
derived (maximum test score = 105 points).

Conclusion: We developed a test for the assessment of clinical competences in basic thoracic ultrasound with solid validity evidence, and a pass/fail standard with no false positives or false negatives.

Objective: To assess the effect of virtual reality training on an actual laparoscopic operation.

Setting and Participants: Seven gynaecological departments in the Zeeland region of Denmark. 24 first and second year registrars specialising in gynaecology and obstetrics.

Interventions: Proficiency based virtual reality simulator training in laparoscopic salpingectomy and standard clinical education (controls).

Main outcome measure: The main outcome measure was technical performance assessed by two independent observers blinded to trainee and training status using a previously validated general and task specific rating scale. The secondary outcome measure was operation time in minutes.

Results: The simulator trained group (n=11) reached a median total score of 33 points (interquartile range 32-36 points), equivalent to the experience gained after 20-50 laparoscopic procedures, whereas the control group (n=10) reached a median total score of 23 (22-27) points, equivalent to the experience gained from fewer than five procedures (P<0.001). The median total operation time in the simulator trained group was 12 minutes (interquartile range 10-14 minutes) and in the control group was 24 (20-29) minutes (P<0.001). The observers’ inter-rater
agreement was 0.79.

Conclusion: Skills in laparoscopic surgery can be increased in a clinically relevant manner using proficiency based virtual reality simulator training. The performance level of novices was increased to that of intermediately experienced laparoscopists and operation time was halved. Simulator training should be considered before trainees carry out laparoscopic procedures.

Background: Virtual reality (VR) training has been shown previously to improve intraoperative performance during part of a laparoscopic cholecystectomy. The aim of this study was to assess the effect of proficiency-based VR training on the outcome of the first 10 entire cholecystectomies performed by novices.

Methods: Thirteen laparoscopically inexperienced residents were randomized to either (1) VR training until a predefined expert level of performance was reached, or (2) the control group.  Videotapes of each resident’s first 10 procedures were reviewed independently in a blinded fashion and scored for predefined errors.

Results: The VR-trained group consistently made significantly fewer errors (P = .0037). On the other hand, residents in the control group made, on average, 3 times as many errors and used 58% longer surgical time. Conclusions: The results of this study show that training on the VR simulator to a level of proficiency significantly improves intraoperative performance during a resident’s first 10 laparoscopic cholecystectomies.

Background: While simulation training has been established as an effective method for improving laparoscopic surgical performance in surgical residents, few studies have focused on its use for attending surgeons, particularly in obstetrics and gynecology. Surgical simulation may have a role in improving and maintaining proficiency in the operating room for practicing obstetrician gynecologists.

Objective: We sought to determine if parameters of performance for validated laparoscopic virtual simulation tasks correlate with surgical volume and characteristics of practicing obstetricians and gynecologists.

Study Design: All gynecologists with laparoscopic privileges (n ¼ 347) from 5 academic medical centers in New York City were required to complete a laparoscopic surgery simulation assessment. The physicians took a presimulation survey gathering physician self-reported characteristics and then performed 3 basic skills tasks (enforced peg transfer, lifting/grasping, and cutting) on the LapSim virtual reality laparoscopic simulator (Surgical Science Ltd, Gothenburg, Sweden). The association between simulation outcome scores (time, efficiency, and
errors) and self-rated clinical skills measures (self-rated laparoscopic skill score or surgical volume category) were examined with regression models.

Results: The average number of laparoscopic procedures per month was a significant predictor of total time on all 3 tasks (P ¼ .001 for peg transfer; P ¼ .041 for lifting and grasping; P < .001 for cutting). Average monthly laparoscopic surgical volume was a significant predictor of 2 efficiency scores in peg transfer, and all 4 efficiency scores in cutting (P¼ .001 to P ¼ .015). Surgical volume was a significant predictor of errors in lifting/grasping and cutting (P < .001 for both). Self-rated laparoscopic skill level was a significant predictor of total time in all 3 tasks (P < .0001 for peg transfer; P ¼ .009 for lifting and grasping; P < .001 for cutting) and a significant predictor of nearly all efficiency scores and errors scores in all 3 tasks.

Conclusion: In addition to total time, there was at least 1 other objective performance measure that significantly correlated with surgical volume for each of the 3 tasks. Higher-volume physicians and those with fellowship training were more confident in their laparoscopic skills. By determining simulation performance as it correlates to active physician practice, further studies may help assess skill and individualize training to maintain skill levels as case volumes fluctuate.

Background: Simulation training is strongly advocated by 24/7 risk-rich professions because swift learning curve inflection point attainment delivers earlier competence; the left-shift effect. The aim of this study was to determine the value of haptic laparoscopic virtual reality simulation, by iterative benchmark exercise (n = 8), before simulated laparoscopic appendicectomy (SLA); the hypothesis was that favorable benchmark learning curve trajectories would be associated with improved SLA competence when compared with consultant expert performance.

Methods: A 28-trainee cohort completed 1349 Laparoscopic Haptic Virtual Reality Skills (LHVRS) tasks, during which 19 ergonomic variables were assessed by virtual interface, including force feedback (Surgicalscience.com), before 153 SLAs. Primary outcome measure was SLA composite competence score related to six consultant trainer experts.

Results: Of the eight LHVRS tasks, the three with the steepest learning curve trajectories correlated with better median overall SLA competence scores, namely tissue grasping/lifting (rho = 0.362, P = .049), fine dissection (rho = 0.388, P = .028), and camera navigation (rho = 0.518, P = .007); fine dissection was the only haptic laparoscopic virtual reality simulation task that predicted a SLA score within a Youden index defined, 70% of the consultant expert level (area under curve [AUC] = 0.803, P = .028). A significant SLA learning curve emerged, with a learning curve trajectory inflection point at the fourth SLA attempt (first SLA 30.5% versus fourth SLA score 76.0%, gradient 76°, P = .010).

Conclusion: Learning curve trajectory can be measured, influenced, and accelerated significantly; a pronounced left-shift effect, with translational potential for enhanced shorter training time and improved patient safety.

Purpose: To analyze the learning curves of ultrasound novices in fetal echocardiography during structured simulation-based ultrasound training (SIM-UT) including a virtual, randomly moving fetus.

Methods: 11 medical students with minimal (< 10 h) prior obstetric ultrasound experience underwent 12 h of structured fetal echocardiography SIM-UT in individual hands-on sessions during a 6-week training program. Their learning progress was assessed with standardized tests after 2, 4, and 6 weeks of SIM-UT. Participants were asked to obtain 11 fetal echocardiography standard planes (in accordance with ISUOG and AHA guidelines) as quickly as possible. All tests were carried out under real life, examination-like conditions on a healthy, randomly moving fetus. Subsequently, we analyzed the rate of correctly obtained images and the total time to completion (TTC). As reference groups, 10 Ob/Gyn physicians (median of 750 previously performed Ob/Gyn scans) and 10 fetal echocardiography experts (median of 15,000 previously performed Ob/Gyn scans) were examined with the same standardized tests.

Results: The students showed a consistent and steady improvement of their ultrasound performance during the training program. After 2 weeks, they were able to obtain > 95% of the standard planes correctly. After 6 weeks, they were significantly faster than the physician group (p < 0.001) and no longer significantly slower than the expert group (p = 0.944).

Conclusion: SIM-UT is highly effective to learn fetal echocardiography. Regarding the acquisition of the AHA/ISUOG fetal echocardiography standard planes, the students were able to reach the same skill level as the expert group within 6 weeks.

Background: Realistic virtual reality (VR) simulators have greatly expanded the tools available for training surgeons and interventionalists. While this technology is effective in improving performance in many fields, it has never been evaluated for neuroendovascular procedures. This study aims to determine whether VR is an effective tool for improving neuroendovascular skill among trainees.

Methods: Trainees performed two VR revascularizations of a right-sided middle cerebral artery (MCA) thrombosis and their times to procedural benchmarks (time to enter internal carotid artery [ICA], traverse clot, and complete procedure) were compared. To determine whether the improvement was case specific, trainees with less procedural exposure were timed during VR left-sided ICA (LICA) aneurysm coiling before or after performing MCA thrombectomy simulations. To determine the value of observing simulations, medical students were timed during the right MCA revascularization simulations after watching other VR procedures.

Results: Trainees significantly improved their time to every procedural benchmark during their second MCA revascularization (mean decrease = 1.08, 1.57, and 2.24 min; P = 0.0072, 0.0466, and 0.0230). In addition, time required to access the LICA during aneurysm coiling was shortened by 0.77 min for each previous VR right MCA revascularization performed (P = 0.0176; r² = 0.71). Finally, medical students’ MCA revascularization simulation times improved by 0.87 min for each prior simulation viewed (P r² = 0.96).

Conclusion: Both performance and viewing of simulated procedures produced significant decreases in time to reach neuroendovascular procedural benchmarks. These data show that VR simulation is a valuable tool for improving trainee skill in neuroendovascular procedures.

Background: Virtual reality (VR) has been recognized as a useful modality in the training of surgical skills. With respect to basic endoscopic skill training of urology, we sought to investigate the effectiveness of the UroMentor(TM) virtual reality simulator (VRS) in the skill acquisition of flexible cystoscopy.

Methods: Urologists familiar with rigid cystoscopy procedures were selected to take part in a virtual training course of flexible cystoscopy. Changes in total operating time, frequency of injury, number of digital markers inside the bladder, and the global rating scale (GRS) scores were assessed following eight repeated training sessions on the UroMentor(TM).

Results: Eighteen urologists voluntarily took part in the study. Total operating time was significantly lower after eight sessions of training by comparison ((111 ± 10) seconds and (511 ± 67) seconds, respectively; P < 0.001). Additionally, the frequency of injury decreased with training from (12 ± 2) times to (5 ± 1) times (P < 0.001), while the number of digital markers observed increased from 9 ± 0 to 10 ± 1 (P = 0.005). Finally, training with the UroMentor(TM) resulted in a GRS increase from (1.3 ± 0.2) points to (3.9 ± 0.2) points (P < 0.001).

Conclusion: The VRS UroMentor(TM) can improve urologists’ ability to perform flexible cystoscopy and could be used as an effective training tool for trainees.

Background and Study Objectives: Operator competency is essential for esophagogastroduodenoscopy (EGD) quality, which makes appropriate training with a final test important. The aims of this study were to develop a test for assessing skills in performing EGD, gather validity evidence for the test, and establish a credible pass/fail score.

Methods: An expert panel developed a practical test using the Simbionix GI Mentor II simulator (3 D Systems) and an EGD phantom (OGI 4, CLA Medical) with a diagnostic (DP) and a technical skills part (TSP) for a prospective validation study. During the test a supervisor measured: 1) total time; 2) degree of mucosal visualization; and 3) landmarks and pathology identification. The contrasting groups standard setting method was used to establish a pass/fail score.

Results: We included 15 novices (N), 10 intermediates (I), and 10 experienced endoscopists (E). The internal structure was high with a Cronbach’s alpha of 0.76 for TSP time consumption and 0.74 for the identification of landmarks.

Mean total times, in minutes, for the DP were N 15.7, I 11.3, and E 7.0, and for TSP., they were N 7.9, I 8.9, and E 2.9. The total numbers of identified landmarks were N 26, I 41, and E 48. Mean visualization percentages were N 80, I 71, and E 71. A pass/fail standard was established requiring identification of all landmarks and performance of the TSP in < 5 minutes. All experienced endoscopists passed, while none of the endoscopists in the other categories did.

Conclusions: We established a test that can distinguish between participants with different competencies. This enables an objective and evidence-based approach to assessment of competencies in EGD.

Introduction: Medical and surgical simulation is in high demand. It is widely used in North America as a method of education and training of medical students and surgical residents. Learning anatomy and vaginal surgery are based on palpation recognition of different structures. The absence of visual control of actions learners is a limiting factor for the reproducibility of surgical techniques prolapse and urinary incontinenence. However, this reproducibility is the only guarantee of success and safety of these minimally invasive surgeries.

Methods: We evaluated the contribution of an educational module perineal anatomy using a system combining anatomic mannequin and a computerized 3D virtual simulator (Pelvic Mentor®, Simbionix) in the knowledge of pelvic-perineal anatomical structures for eight residents of obstetrics and gynecology hospitals in Paris.

Results: The self-study training module has led to substantial improvements in internal rating with a proportion of structures recognized from 31.25 to 87.5 % (P<0.001) for the front compartment and 20 to 85 % (P<0.001) for the posterior compartment.

Conclusion: The preliminary results suggest that the 3D virtual simulator enhances and facilitates learning the anatomy of the pelvic floor.

Background: After the American Board of Surgery announcement of the Flexible Endoscopy Curriculum requirement in 2014, we implemented a dedicated endoscopy rotation at the post graduate year (PGY)2 level including a simulation curriculum for Fundamentals of Endoscopic Surgery skills. Here we evaluate the outcomes of this implementation.

Methods: Beginning in 2015, we developed a clinical endoscopy and simulation-based rotation to prepare for Fundamentals of Endoscopic Surgery testing. Originally, our curriculum was based on the published Texas Association of Surgical Skills Laboratories curriculum using the GI Mentor and transitioned to a mastery learning curriculum using the Endoscopy Training System in 2016. We evaluated the success of the curriculum in terms of first-time pass rates, training time required, and comparison to previously published benchmarks based on clinical experience.

Results: Since 2015, a total of 37 general surgery residents in our program were Fundamentals of Endoscopic Surgery tested (PGY2 = 24, PGY3 = 4, PGY5 = 9); 84% (31) completed the Endoscopy Training System curriculum. At the time of testing, 73% (27) had performed <25 esophagogastroduodenoscopies, and 46% had performed <25 colonoscopies. Ninety-two percent (34) spent 10 hours or less completing the curriculum. The first-time pass rate for those completing the Endoscopy Training System curriculum was 97% vs 67% for those not completing the Endoscopy Training System curriculum (P = .01). For residents completing the Endoscopy Training System curriculum, total Fundamentals of Endoscopic Surgery scores were discernibly higher (472 vs 389, P < .01), as were 3/5 task scores (Nav1 80 vs 67, P = .02; Loop2 36 vs 8, P = .02; Retro3 89 vs 71, P = .02). Despite clinical inexperience (<25 esophagogastroduodenoscopies and <50 colonoscopies), PGY2s yielded a mean score of 454 and a pass rate of 92%. This was similar to PGY5s (427, 89%; P = .3) and compares to benchmark data of endoscopists with >300 cases.

Conclusion: Early implementation of flexible endoscopy training with a simulation-based curriculum results in Fundamentals of Endoscopic Surgery performance equal to a clinical experience not often gained during surgical residency. Often requiring <10 hours, this represents a fantastic return on investment for this training.

Introduction: Decreases in trainees’ working hours, coupled with evidence of worse outcomes when hip arthroscopies are performed by inexperienced surgeons, mandate an additional means of training. Though virtual reality simulation has been adopted by other surgical specialties, its slow uptake in arthroscopic training is due to a lack of evidence as to its benefits. These benefits can be demonstrated through learning curves associated with simulator training—with practice reflecting increases in validated performance metrics.

Methods: Twenty-five medical students with no previous experience of hip arthroscopy completed seven weekly simulated arthroscopies of a healthy virtual hip joint using a 70° arthroscope in the supine position. Twelve targets were visualised within the central compartment, six via the anterior portal, three via the anterolateral portal and three via the posterolateral portal. Task duration, number of collisions (bone and soft-tissue), and distance travelled by arthroscope were measured by the simulator for every session of each student.

Results: Learning curves were demonstrated by the students, with improvements in time taken, number of collisions (bone and soft-tissue), collision length and efficiency of movement (all p<0.01). Improvements in time taken, efficiency of movement and number of collisions with soft-tissue were first seen in session 3 and improvements in all other parameters were
seen in session 4. No differences were found after session 5 for time taken and length of soft-tissue collision. No differences in number of collisions (bone and soft-tissue), length of collisions with bone, and efficiency of movement were found after session 6.

Conclusions: The results of this study demonstrate learning curves for a hip arthroscopy simulator, with significant improvements seen after three sessions. All performance metrics were found to improved, demonstrating sufficient visuo-haptic consistency within the virtual environment, enabling individuals to develop basic arthroscopic skills.

Purpose: To evaluate the correlation between timed task performance on an arthroscopy shoulder simulator and participation in a standardized expert shoulder arthroscopy educational course.

Methods: Orthopaedic trainees were voluntarily recruited from over 25 residency programs throughout the United States and Canada. Each trainee was tested on arrival at the Arthroscopy Association of North America orthopaedic learning center on a virtual reality arthroscopy shoulder simulator, and his or her performance was objectively scored. Each trainee’s postgraduate year level was recorded, as was his or her experience in residency with shoulder arthroscopy as measured by Accreditation Council for Graduate Medical Education case-log totals. After the focused 4-day training curriculum consisting of didactics and cadaveric experience, each trainee was re-evaluated on the same simulator. Statistical analysis was performed to determine if participation in the course was associated with changes in simulation performance from before to after assessment.

Results: Forty-eight trainees completed the testing. On completion of the course, trainees showed significant improvements in all objective measures recorded by the simulator. Total probe distance needed to complete the task decreased by 42% (from 420.4 mm to 245.3 mm, P < .001), arthroscope tip distance traveled decreased by 59% (from 194.1 mm to
80.2 mm, P < .001), and time to completion decreased by 38% (from 66.8 seconds to 41.6 seconds, P < .001). Highly
significant improvements in all 3 measures suggest improved instrument handling, anatomic recognition, and
arthroscopy-related visual-spatial ability.

Conclusions: This study shows objective improvement in orthopaedic trainee basic arthroscopy skill and proficiency after a standardized 4-day arthroscopy training curriculum. The results validate the Arthroscopy Association of North America resident training course and its curriculum with objective evidence of benefit. Level of Evidence: Level III, prospective study of nonconsecutive participants.

Objective and Background: This study aimed to determine the cost-effectiveness of a Proficiency based Stepwise Endovascular Curricular Training (PROSPECT) program, including e-learning and hands on virtual reality simulation.

Methods: A prospective, single blinded, randomised controlled trial (RCT) was carried out to evaluate the impact of a PROSPECT training program on real life operative performance. Under supervision, all subjects performed two endovascular interventions on patients with symptomatic iliac and/or superficial femoral artery stenosis. Primary outcomes were technical performance (Global Rating Scale, Examiner Checklist), operative metrics, and patient outcomes, adjusted for case difficulty and the trainee’s experience. Additionally, an analysis of costs and savings related to implementation of this endovascular training program was performed. Thirty-two general surgery trainees were randomised into three groups: group 1 (n = 11) received e-learning and simulation training (PROSPECT program); group 2 (n = 10) only had access to e-learning; group 3 (n = 11) did not receive supplementary education besides clinical training. Developmental cost, implementation cost, training time cost, and the operational cost of PROSPECT were determined. Time spent studying and practicing was converted to indirect saving of operating time. The costs of logistics, faculty time supervising simulation sessions, and 30 day complication rates were registered. Sensitivity analysis was performed to assess the robustness of the results.

Results: Fifty-eight peripheral endovascular interventions, performed by 29 surgical trainees (three dropouts) were included in this RCT from October 2014 to February 2016. Annual costs from the perspective of the hospital were €6589 for curriculum design, €31,484 for implementation, and €1143 in operational costs. Per trainee, simulation-based training until proficiency cost €3806. In comparison, if endovascular proficiency levels were obtained with conventional training only, this may have cost €5001 per trainee.

Conclusion: Simulation based training in endovascular procedures may be cost saving, because training occurs outside the angiosuite. It is possible that cost savings are underestimated as, in contrast to the literature, prevented costs related to complications could not be defined.

Background: Full-procedure virtual reality (VR) simulator training in robotic-assisted radical prostatectomy (RARP) is a new tool in surgical education.

Methods: Description of the development of a VR RARP simulation model, (RobotiX-Mentor®) including non-guided bladder neck (ngBND) and neurovascular bundle dissection (ngNVBD) modules, and assessment of face, content, and construct validation of the ngBND and ngNVBD modules by robotic surgeons with different experience levels.

Results: Simulator and ngBND/ngNVBD modules were rated highly by all surgeons for realism and usability as a training tool. In the ngBND-task construct, validation was not achieved in task-specific performance metrics. In the ngNVBD, task-specific performance of the expert/intermediately experienced surgeons was significantly better than that of novices.

Conclusions: We proved face and content validity of simulator and both modules, and construct validity for generic metrics of the ngBND module and for generic and task-specific metrics of the ngNVBD module.

Objectives: To compare the contribution of training using virtual simulation with theoretical instruction alone in the learning of pelvic ultrasound for gynecological emergencies in non-experienced students.

Methods: A unicentre randomised controlled trial was conducted. The participants were students in the last year of medical studies during their internship in the obstetrics and gynecology department at the University Hospital Center of Angers. Twenty participants were randomised to a 1:1 ratio in the training group and control group of 10 students each. All participants received a two hours theoretical course on the physical basis of ultrasound, the use of probes, as well as the basic principles of an trans-vaginal ultrasound examination centreed on the different standardised images to be given to gynecologic emergencies. For the training group an additional 20-minute pelvic ultrasound training on a trans-vaginal virtual simulator was performed. The participants were evaluated on images taken from patients consulting our gynecological emergencies centre. The pictures evaluated were sagittal and coronal sections of the uterus, left and right ovarian sections and a Morrison pouch view. The primary outcome was based on picture quality scores described by Salomon et al. with the French Society for the Improvement of Ultrasound Practices (SFAPE) and by Popowski et al. for Standardisation Acute Female Echography (SAFE). Secondaries outcomes focused on the general ultrasound skills and the duration of acquisition of the pictures.

Results: The mean SFAPE and SAFE scores were significantly higher in the training group than in the control group (14.5 ± 3.1, p = 0.046 and 10.1 ± 2.08, p = 0.016, respectively).

Conclusions: After a very short training session, using virtual simulation, inexperienced students produce proper pictures for the management of gynecological emergencies and better quality ultrasound pictures than students trained only theoretically.

Background: Clinical lung ultrasound (LUS) is a fast bedside diagnostic tool which can assist clinicians in decisions regarding the treatment and monitoring of patients with respiratory symptoms. LUS training and education differ widely, and is often done in a clinical setting, with potential risks for patients if decisions are made based on the wrong interpretations. No clear guidelines or recommendations for objective and standardized assessment of LUS skills exist, and those that do are often based on a fixed time-frame or an arbitrary number of examinations performed; this does not ensure adequate competencies.

Objectives: The study aimed to develop and gather validity evidence for a practical, simulation-based test in LUS.

Methods: Nine cases were developed in collaboration with 3D Systems Healthcare, Littleton, CO, USA, representing the most common diagnosis and sonographic findings in patients with respiratory symptoms. Thirty-six participants with different levels of competence in LUS, completed the test. The participants were divided into groups, i.e., novices, intermediates, and experienced, according to their experience with LUS, the number of examinations they had performed, and any research they had conducted. Their answers were used for item analyses.

Results: The intraclass correlation coefficient, Cronbachs’ α, was 0.69 summarized, and there was a statistically significant difference (p < 0.001) between the novices and the trained participants (intermediates and experienced). A pass/fail score of 16 points was calculated according to the contrasting-groups method.

Conclusion: We developed a test for the assessment of clinical competencies in LUS. The test proved solid validity evidence, and a pass/fail standard without any false-negatives, and only 2 explained false-positives.

Purpose: To investigate validity evidence for a simulator-based test in robot-assisted radical prostatectomy (RARP).

Materials and Methods: The test consisted of three modules on the RobotiX Mentor VR-simulator: Bladder Neck Dissection, Neurovascular Bundle Dissection, and Urethrovesical Anastomosis. Validity evidence was investigated by using Messick’s framework by including doctors with different RARP experience: novices (who had assisted for RARP), intermediates (robotic surgeons, but not RARP surgeons), or experienced (RARP surgeons). The simulator metrics were analyzed, and Cronbach’s alpha and generalizability theory were used to explore reliability. Intergroup comparisons were done with mixed-model, repeated measurement analysis of variance and the correlation between the number of robotic procedures and the mean test score were examined. A pass/fail score was established by using the contrasting groups’ method.

Results: Ten novices, 11 intermediates, and 6 experienced RARP surgeons were included. Six metrics could discriminate between groups and showed acceptable internal consistency reliability, Cronbach’s alpha = 0.49, p < 0.001. Test–retest reliability was 0.75, 0.85, and 0.90 for one, two, and three repetitions of tests, respectively. Six metrics were combined into a simulator score that could discriminate between all three groups, p = 0.002, p < 0.001, and p = 0.029 for novices vs intermediates, novices vs experienced, and intermediates vs experienced, respectively. Total number of robotic operations and the mean score of the three repetitions were significantly correlated, Pearson’s r = 0.74, p < 0.001.

Conclusion: This study provides validity evidence for a simulator-based test in RARP. We determined a pass/fail level that can be used to ensure competency before proceeding to supervised clinical training.

Objective: To present the initial experience of the first tier of surgeons trained in the new model of robotic surgery training proposed by the CBC.

Methods: We retrospectively collected data and information on training with the Da Vinci SI robotic system. The variables analyzed were, in the pre-clinical phase, time of completion of each step by surgeon and number of hours in the simulator, and in the clinical phase, operations carried out by the training group, number of surgeons who performed nine procedures in ninety days (“9 in 90”), time of docking, time of console, and results surgical.

Results: We interviewed 39 surgeons before training started; 20 (51.3%) reached the clinical phase. The average age of surgeons was 47.9 years (38-62). The average time between the first interview and the delivery of the online certificate was 64 days (15-133). The surgeons have made an average of 51h and 36 minutes of robot simulation (40-83 hours). The total number of cases in which the training surgeons participated as first assistant was 418, with an average of 20.9 per surgeon. The time of pre-clinical training had an average of 116 days (48-205).

Conclusion: The new model proposed had good acceptance by all surgeons trained and proved safe in the initial sample.

Background

While laparoscopic assistance is often entrusted to less experienced individuals, such as residents, medical students, and operating room nurses, it is important to note that they typically receive little to no formal laparoscopic training. This deficiency can lead to poor visibility during minimally invasive surgery, thus increasing the risk of errors. Moreover, operating room nurses and medical students are currently not included as key users in structured laparoscopic training programs.

Objectives

The aim of this study is to evaluate the laparoscopic skills of OR nurses, clinical medical postgraduate students, and residents before and after undergoing virtual reality training. Additionally, it aimed to compare the differences in the laparoscopic skills among different groups (OR nurses/Students/Residents) both before and after virtual reality training.

Methods

Operating room nurses, clinical medical postgraduate students and residents from a tertiary Grade A hospital in China in March 2022 were selected as participants. All participants were required to complete a laparoscopic simulation training course in 6 consecutive weeks. One task from each of the four training modules was selected as an evaluation indicator. A before-and-after self-control study was used to compare the basic laparoscopic skills of participants, and laparoscopic skill competency was compared between the groups of operating room nurses, clinical medical postgraduate students, and residents.

Results

Twenty-seven operating room nurses, 31 clinical medical postgraduate students, and 16 residents were included. The training course scores for the navigation training module, task training module, coordination training module, and surgical skills training module between different groups (operating room nurses/clinical medical postgraduate/residents) before laparoscopic simulation training was statistically significant (p < 0.05). After laparoscopic simulation training, there was no statistically significant difference in the training course scores between the different groups. The surgical level scores before and after the training course were compared between the operating room nurses, clinical medical postgraduate students, and residents and showed significant increases (p < 0.05).

Conclusion

Our findings show a significant improvement in laparoscopic skills following virtual surgery simulation training across all participant groups. The integration of virtual surgery simulation technology in surgical training holds promise for bridging the gap in laparoscopic skill development among health care professionals.

Background: Simulation-based bronchoscopy training is increasingly used, but effectiveness remains uncertain. We sought to perform a comprehensive synthesis of published work on simulation-based bronchoscopy training.

Methods: We searched MEDLINE, EMBASE, CINAHL, PsycINFO, ERIC, Web of Science, and Scopus for eligible articles through May 11, 2011. We included all original studies involving health professionals that evaluated, in comparison with no intervention or an alternative instructional approach, simulation-based training for flexible or rigid bronchoscopy. Study selection and data abstraction were performed independently and in duplicate. We pooled results using random effects meta-analysis.

Results: From an initial pool of 10,903 articles, we identified 17 studies evaluating simulation-based bronchoscopy training. In comparison with no intervention, simulation training was associated with large benefits on skills and behaviors (pooled effect size, 1.21 [95% CI, 0.82-1.60]; n=8 studies) and moderate benefits on time (0.62 [95% CI, 0.12-1.13]; n=7). In comparison with clinical instruction, behaviors with real patients showed nonsignificant effects favoring simulation for time (0.61 [95% CI, -1.47 to 2.69]) and process (0.33 [95% CI, -1.46 to 2.11]) outcomes (n=2 studies each), although variation in training time might account for these differences. Four studies compared alternate simulation-based training approaches. Inductive analysis to inform instructional design suggested that longer or more structured training is more effective, authentic clinical context adds value, and animal models and plastic part-task models may be superior to more costly virtual-reality simulators.

Conclusions: Simulation-based bronchoscopy training is effective in comparison with no intervention. Comparative effectiveness studies are few.

Methods: 26 novice participants were randomised to either procedural VR (n = 13) or basic VR simulation (n = 13). Both cohorts completed a structured training program. Simulator metric data were used to plot learning curves. All participants then performed parts of a robotic-assisted radical prostatectomy (RARP) on a fresh frozen cadaver. Performances were compared against a cohort of 9 control participants without any training experience. Performances were video recorded and assessed blindly using GEARS post hoc.

Results: Learning curve analysis demonstrated improvements in technical skill for both training modalities although procedural training was associated with greater training effects. Any VR training resulted in significantly higher GEARS scores than no training (GEARS score 11.3 ± 0.58 vs. 8.8 ± 2.9, p = 0.002). Procedural VR training was found to be more effective than both basic VR training and no training (GEARS 11.9 ± 2.9 vs. 10.7 ± 2.8 vs. 8.8 ± 1.4, respectively, p = 0.03).

Purpose: To evaluate the correlation between timed task performance on an arthroscopy shoulder simulator and participation in a standardized expert shoulder arthroscopy educational course.

Methods: Orthopaedic trainees were voluntarily recruited from over 25 residency programs throughout the United States and Canada. Each trainee was tested on arrival at the Arthroscopy Association of North America orthopaedic learning center on a virtual reality arthroscopy shoulder simulator, and his or her performance was objectively scored. Each trainee’s postgraduate year level was recorded, as was his or her experience in residency with shoulder arthroscopy as measured by Accreditation Council for Graduate Medical Education case-log totals. After the focused 4-day training curriculum consisting of didactics and cadaveric experience, each trainee was re-evaluated on the same simulator. Statistical analysis was performed to determine if participation in the course was associated with changes in simulation performance from before to after assessment.

Results: Forty-eight trainees completed the testing. On completion of the course, trainees showed significant improvements in all objective measures recorded by the simulator. Total probe distance needed to complete the task decreased by 42% (from 420.4 mm to 245.3 mm, P < .001), arthroscope tip distance traveled decreased by 59% (from 194.1 mm to 80.2 mm, P < .001), and time to completion decreased by 38% (from 66.8 seconds to 41.6 seconds, P < .001). Highly significant improvements in all 3 measures suggest improved instrument handling, anatomic recognition, and arthroscopy-related visual-spatial ability.

Conclusions: This study shows objective improvement in orthopaedic trainee basic arthroscopy skill and proficiency after a standardized 4-day arthroscopy training curriculum. The results validate the Arthroscopy Association of North America resident training course and its curriculum with objective evidence of benefit.

Level of evidence: Level III, prospective study of nonconsecutive participants.

Background:  Surgical simulation has become increasingly relevant to orthopaedic surgery education and could translate to improved operating room proficiency in orthopaedic surgery trainees.

Purpose: To compare the arthroscopic performance of junior orthopaedic surgery residents who received training with a knee and shoulder arthroscopy surgical simulator with those who received didactic training.
Study Design: Controlled laboratory study.

Methods: Fourteen junior orthopaedic surgery residents at a single institution were randomized to receive knee and shoulder arthroscopy training with a surgical simulator (n = 8) or didactic lectures with arthroscopy models (n = 6). After their respective training, performance in diagnostic knee and shoulder arthroscopy was assessed using a cadaveric model. Time to completion and assessment of arthroscopic handling using a subjective injury grading index (scale, 1-10) was then used to evaluate performance
in final cadaveric testing.

Results: Orthopaedic surgery residents who trained with a surgical simulator outperformed the didactic-trained residents in shoulder arthroscopy by time to completion (–35%; P = .02) and injury grading index (–35%; P = .01). In addition, a trend toward improved performance of knee arthroscopy by the simulator-trained group was found by time to completion (–36%; P = .09) and injury grading index (P = .08).

Conclusion: In this study, junior orthopaedic surgery residents who trained with a surgical simulator demonstrated improved arthroscopic performance in both knee and shoulder arthroscopy. However, future validation of surgical simulator training for orthopaedic surgery residents remains warranted.

Clinical Relevance: Surgical skill development with an arthroscopy surgical simulator could translate to improved arthroscopy performance in the operating room.

Introduction and Objectives: Improvements in virtual reality (VR) technology have enabled the development of procedural simulation training, which closely replicate surgical procedures. VR simulation training has been shown to be highly effective for robotic surgical training, however to-date curricula are limited to generic basic skills training. This RCT aims to compare the transfer of learning following procedural VR or standard basic skills VR training.

Methods: Initially 25 novice surgeons underwent basic robotic skills training, completing three FRS tasks. Participants were then block randomised to standard basic VR training or procedural VR training. All training was performed on the RobotiX Mentor (3D Systems (Airport City, Israel) VR robotic simulator. Standard basic skills training comprised further training following the FRS curriculum. The procedural simulation group underwent training on the guided bladder neck dissection and guided urethrovesical anastomosis tasks, parts of radical prostatectomy training module. Both groups completed a total of at least 5 hours of training. Following training both groups underwent transfer of skills assessment on fresh frozen cadavers using a Da Vinci Xi surgical robot in a simulated operating room environment. Their performances were compared to a control group of novice training without training. All performances were video recorded and were assessed blindly post hoc by a trained expert using GEARS.

Results: Baseline FRS scores were equal between the two groups (p=0.5). Subjects in both arms completed an average of 5.6 ± 0.3 hours of training. VR training (basic or procedural) resulted in a significantly higher GEARS score than no training, (mean GEARS score 11.3 ± 0.6 vs 8.8 ± 2.9 p=0.002). Procedural training resulted in significantly higher GEARS score than either basic training or control (p=0.03)(Figure 1).

Conclusions: This study provides the first evidence supporting the use of procedural-based VR simulation for training robotic skills even in novice participants. It also provides further validity evidence to support the use of VR simulation and the effective transfer of learned skills.

Design: Participants completed the bladder neck dissection task and urethrovesical anastomosis task (UVA) as part of the prostatectomy module. Surgeons completed feedback questionnaires assessing the realism, content, acceptability and feasibility of the module. Novice surgeons completed a 5.5-hour training programme using both tasks.

Setting: King’s College London, London

Participants: 13 novice, 24 intermediate and 8 expert surgeons completed the validation study.

Results: Realism was scored highly for BDN (mean 3.4/5) and UVA (3.74/5), as was importance of BDN (4.32/5) and UVA (4.6/5) for training. It was rated as a feasible (3.95/5) and acceptable (4/5) tool for training. Experts performed significantly better than novice group in 6 metrics in the UVA including time (p = 0.0005), distance by camera (p = 0.0010) and instrument collisions (p = 0.0033), as well as task-specific metrics such as number of unnecessary needle piercing points (p = 0.0463). In novice surgeons, a significant improvement in performance after training was seen in many metrics for both tasks. For bladder neck dissection task, this included time (p < 0.0001), instrument collisions (p = 0.0013) and total time instruments are out of view (p = 0.0251). For UVA, this included time (p = 0.0135), instrument collisions (p = 0.0066) and task-specific metrics such as injury to the urethra (p = 0.0032) and bladder (p = 0.0189).

Background: Very few studies have addressed the transferability of skills from virtual reality (VR) to real life. The aim of this study was to assess the feasibility and effectiveness of teaching intracorporeal knot tying (ICKT) by VR simulation only.

Methods: Twenty novices underwent structured training of basic skills training on the Minimally Invasive Surgical Trainer simulator (Mentice AB, Gothenburg, Sweden) followed by knot tying training on the LapSim simulator (Surgical Science, Gothenburg, Sweden). They were assessed pre- and post-training on a video trainer. Assessment of performance included motion tracking and video-based checklist. Nonparametric statistical analysis was used, and P < .05 was deemed significant.

Results: All participants completed a correct knot as compared with only 25% before VR training. Time to completion was 66% faster and knot quality 45% better after VR training. Significant reduction in number of movements (P = .006) and distance traveled (P < .000) by both hands after VR training.

Conclusions: Teaching ICKT by VR simulators only is feasible and effective. Furthermore, this study highlights the complementary use of different VR simulators within a structured curriculum.