400+ Validation Studies

Background: Virtual reality (VR) simulators are increasingly being used for surgical skills training. It is unclear what skills are best improved via VR, translate to live surgical skills, and influence patient outcomes.

Objective: To assess surgeons in VR and live surgery using a suturing assessment tool and evaluate the association between technical skills and a clinical outcome.

Design, setting, and participants: This prospective five-center study enrolled participants who completed VR suturing exercises and provided live surgical video. Graders provided skill assessments using the validated End-To-End Assessment of Suturing Expertise (EASE) suturing evaluation tool.

Outcome measurements and statistical analysis: A hierarchical Poisson model was used to compare skill scores among cohorts and evaluate the association of scores with clinical outcomes. Spearman’s method was used to assess correlation between VR and live skills.

Results and limitations: Ten novices, ten surgeons with intermediate expertise (median 64 cases, interquartile range [IQR] 6-80), and 26 expert surgeons (median 850 cases, IQR 375-3000) participated in this study. Intermediate and expert surgeons were significantly more likely to have ideal scores in comparison to novices for the subskills needle hold angle, wrist rotation, and wrist rotation needle withdrawal (p < 0.01). For both intermediate and expert surgeons, there was positive correlation between VR and live skills for needle hold angle (p < 0.05). For expert surgeons, there was a positive association between ideal scores for VR needle hold angle and driving smoothness subskills and 3-mo continence recovery (p < 0.05). Limitations include the size of the intermediate surgeon sample and clinical data limited to expert surgeons.

Conclusions: EASE can be used in VR to identify skills to improve for trainee surgeons. Technical skills that influence postoperative outcomes may be assessable in VR.

Andrea Moglia, Luca Morelli, Vincenzo Ferrari, Mauro Ferrari, Franco Mosca, Alfred Cuschieri

ABSTRACT

Background: There is an increasing interest for a test assessing objectively the innate aptitude for surgery as a craft specialty to complement the current selection process of surgical residents. The aim of this study was to quantify the size of individuals with high, average, and low level of innate psychomotor skills among medical students.

Methods: A volunteer sample of 155 medical students, without prior experience with surgical simulator, executed five tasks at a virtual simulator for robot-assisted surgery. They had to reach proficiency twice consecutively in each before moving to the next one. A weighting based on time and number of attempts needed to reach proficiency was assigned to each task.

Results: Nine students (5.8%) out of 155 significantly outperformed all the others on median (i.q.r.) weighted time [44.7 (42.2–47.3) min vs. 98.5 (70.8–131.8) min, p < 0.001], and number of attempts to reach proficiency [14 (12–15) vs. 23 (19–32.75), p < 0.001). Seventeen students (11.0%) scored significantly much worse than the rest on median weighted time [202.2 (182.5–221.0) min vs. 84.3 (65.7–114.4) min, p < 0.001], and number of attempts [42 (40–48) vs. 22 (17.25–28), p < 0.001]. Low correlation between simulator scores and extracurricular activities, like videogames and musical instruments, was found.

Conclusions: The test successfully identified two groups straddling the large cohort with average innate aptitude for psychomotor skills: (i) innately gifted and (ii) with scarce level. Hence, exercises on a virtual simulator are a valid test of innate manual dexterity and can be considered to complement the selection process for a surgical training program, primarily to identify individuals with low innate aptitude for surgery and advise them to consider specialization in other (non-craft) medical specialties.

Background:   Robotic surgery simulation may provide a way for surgeons to acquire specific robotic surgical skills without practicing on live patients.

Methods:   Five robotic surgery experts performed 10 simulator skills to the best of their ability, and thus, established expert benchmarks for all parameters of these skills. A group of credentialed gynecologicsurgeons naive to robotics practiced the simulator skills until theywere able to perform each one as well as our experts. Within a weekof doing so, they completed robotic pig laboratory training, after whichthey performed supracervical hysterectomies as their first-ever livehuman robotic surgery. Time, blood loss, and blinded assessments ofsurgical skill were compared among the experts, novices, and a groupof control surgeons who had robotic privileges but no simulator expo-sure. Sample size estimates called for 11 robotic novices to achieve90% power to detect a 1 SD difference between operative times ofexperts and novices (> = 0.05).

Results:   Fourteen novice surgeons completed the studyVspendingan average of 20 hours (range, 9.7Y38.2 hours) in the simulation lab-oratory to pass the expert protocol. The mean operative times for the ex-pert and novices were 20.2 (2.3) and 21.7 (3.3) minutes, respectively(P = 0.12; 95% confidence interval, j1.7 to 4.7), whereas the mean timefor control surgeons was 30.9 (0.6) minutes (P G 0.0001; 95% confi-dence interval, 6.3Y12.3). Comparisons of estimated blood loss (EBL) andblinded video assessment of skill yielded similar differences betweengroups.

Conclusions:   Completing this protocol of robotic simulator skills translated to expert-level surgical times during live human surgery. As such, we have established predictive validity of this protocol.