400+ Validation Studies

Background: Full-procedure virtual reality (VR) simulator training in robotic-assisted radical prostatectomy (RARP) is a new tool in surgical education.

Methods: Description of the development of a VR RARP simulation model, (RobotiX-Mentor®) including non-guided bladder neck (ngBND) and neurovascular bundle dissection (ngNVBD) modules, and assessment of face, content, and construct validation of the ngBND and ngNVBD modules by robotic surgeons with different experience levels.

Results: Simulator and ngBND/ngNVBD modules were rated highly by all surgeons for realism and usability as a training tool. In the ngBND-task construct, validation was not achieved in task-specific performance metrics. In the ngNVBD, task-specific performance of the expert/intermediately experienced surgeons was significantly better than that of novices.

Conclusions: We proved face and content validity of simulator and both modules, and construct validity for generic metrics of the ngBND module and for generic and task-specific metrics of the ngNVBD module.

Purpose: To investigate validity evidence for a simulator-based test in robot-assisted radical prostatectomy (RARP).

Materials and Methods: The test consisted of three modules on the RobotiX Mentor VR-simulator: Bladder Neck Dissection, Neurovascular Bundle Dissection, and Urethrovesical Anastomosis. Validity evidence was investigated by using Messick’s framework by including doctors with different RARP experience: novices (who had assisted for RARP), intermediates (robotic surgeons, but not RARP surgeons), or experienced (RARP surgeons). The simulator metrics were analyzed, and Cronbach’s alpha and generalizability theory were used to explore reliability. Intergroup comparisons were done with mixed-model, repeated measurement analysis of variance and the correlation between the number of robotic procedures and the mean test score were examined. A pass/fail score was established by using the contrasting groups’ method.

Results: Ten novices, 11 intermediates, and 6 experienced RARP surgeons were included. Six metrics could discriminate between groups and showed acceptable internal consistency reliability, Cronbach’s alpha = 0.49, p < 0.001. Test–retest reliability was 0.75, 0.85, and 0.90 for one, two, and three repetitions of tests, respectively. Six metrics were combined into a simulator score that could discriminate between all three groups, p = 0.002, p < 0.001, and p = 0.029 for novices vs intermediates, novices vs experienced, and intermediates vs experienced, respectively. Total number of robotic operations and the mean score of the three repetitions were significantly correlated, Pearson’s r = 0.74, p < 0.001.

Conclusion: This study provides validity evidence for a simulator-based test in RARP. We determined a pass/fail level that can be used to ensure competency before proceeding to supervised clinical training.

Objective: To present the initial experience of the first tier of surgeons trained in the new model of robotic surgery training proposed by the CBC.

Methods: We retrospectively collected data and information on training with the Da Vinci SI robotic system. The variables analyzed were, in the pre-clinical phase, time of completion of each step by surgeon and number of hours in the simulator, and in the clinical phase, operations carried out by the training group, number of surgeons who performed nine procedures in ninety days (“9 in 90”), time of docking, time of console, and results surgical.

Results: We interviewed 39 surgeons before training started; 20 (51.3%) reached the clinical phase. The average age of surgeons was 47.9 years (38-62). The average time between the first interview and the delivery of the online certificate was 64 days (15-133). The surgeons have made an average of 51h and 36 minutes of robot simulation (40-83 hours). The total number of cases in which the training surgeons participated as first assistant was 418, with an average of 20.9 per surgeon. The time of pre-clinical training had an average of 116 days (48-205).

Conclusion: The new model proposed had good acceptance by all surgeons trained and proved safe in the initial sample.

Methods: 26 novice participants were randomised to either procedural VR (n = 13) or basic VR simulation (n = 13). Both cohorts completed a structured training program. Simulator metric data were used to plot learning curves. All participants then performed parts of a robotic-assisted radical prostatectomy (RARP) on a fresh frozen cadaver. Performances were compared against a cohort of 9 control participants without any training experience. Performances were video recorded and assessed blindly using GEARS post hoc.

Results: Learning curve analysis demonstrated improvements in technical skill for both training modalities although procedural training was associated with greater training effects. Any VR training resulted in significantly higher GEARS scores than no training (GEARS score 11.3 ± 0.58 vs. 8.8 ± 2.9, p = 0.002). Procedural VR training was found to be more effective than both basic VR training and no training (GEARS 11.9 ± 2.9 vs. 10.7 ± 2.8 vs. 8.8 ± 1.4, respectively, p = 0.03).

Introduction and Objectives: Improvements in virtual reality (VR) technology have enabled the development of procedural simulation training, which closely replicate surgical procedures. VR simulation training has been shown to be highly effective for robotic surgical training, however to-date curricula are limited to generic basic skills training. This RCT aims to compare the transfer of learning following procedural VR or standard basic skills VR training.

Methods: Initially 25 novice surgeons underwent basic robotic skills training, completing three FRS tasks. Participants were then block randomised to standard basic VR training or procedural VR training. All training was performed on the RobotiX Mentor (3D Systems (Airport City, Israel) VR robotic simulator. Standard basic skills training comprised further training following the FRS curriculum. The procedural simulation group underwent training on the guided bladder neck dissection and guided urethrovesical anastomosis tasks, parts of radical prostatectomy training module. Both groups completed a total of at least 5 hours of training. Following training both groups underwent transfer of skills assessment on fresh frozen cadavers using a Da Vinci Xi surgical robot in a simulated operating room environment. Their performances were compared to a control group of novice training without training. All performances were video recorded and were assessed blindly post hoc by a trained expert using GEARS.

Results: Baseline FRS scores were equal between the two groups (p=0.5). Subjects in both arms completed an average of 5.6 ± 0.3 hours of training. VR training (basic or procedural) resulted in a significantly higher GEARS score than no training, (mean GEARS score 11.3 ± 0.6 vs 8.8 ± 2.9 p=0.002). Procedural training resulted in significantly higher GEARS score than either basic training or control (p=0.03)(Figure 1).

Conclusions: This study provides the first evidence supporting the use of procedural-based VR simulation for training robotic skills even in novice participants. It also provides further validity evidence to support the use of VR simulation and the effective transfer of learned skills.

Design: Participants completed the bladder neck dissection task and urethrovesical anastomosis task (UVA) as part of the prostatectomy module. Surgeons completed feedback questionnaires assessing the realism, content, acceptability and feasibility of the module. Novice surgeons completed a 5.5-hour training programme using both tasks.

Setting: King’s College London, London

Participants: 13 novice, 24 intermediate and 8 expert surgeons completed the validation study.

Results: Realism was scored highly for BDN (mean 3.4/5) and UVA (3.74/5), as was importance of BDN (4.32/5) and UVA (4.6/5) for training. It was rated as a feasible (3.95/5) and acceptable (4/5) tool for training. Experts performed significantly better than novice group in 6 metrics in the UVA including time (p = 0.0005), distance by camera (p = 0.0010) and instrument collisions (p = 0.0033), as well as task-specific metrics such as number of unnecessary needle piercing points (p = 0.0463). In novice surgeons, a significant improvement in performance after training was seen in many metrics for both tasks. For bladder neck dissection task, this included time (p < 0.0001), instrument collisions (p = 0.0013) and total time instruments are out of view (p = 0.0251). For UVA, this included time (p = 0.0135), instrument collisions (p = 0.0066) and task-specific metrics such as injury to the urethra (p = 0.0032) and bladder (p = 0.0189).