400+ Validation Studies

Introduction: Recent advancements in surgical technology, reduced working hours, and training opportunities exacerbated by the COVID-19 pandemic have led to an increase in simulation-based training. Furthermore, a rise in endovascular procedures has led to a requirement for high-fidelity simulators that offer comprehensive feedback. This review aims to identify vascular surgery simulation models and assess their validity and levels of effectiveness (LoE) for each model in order to successfully implement them into current training curricula.

Methods: PubMed and EMBASE were searched on January 1, 2021, for full-text English studies on vascular surgery simulators. Eligible articles were given validity ratings based on Messick’s modern concept of validity alongside an LoE score according to McGaghie’s translational outcomes.

Results: Overall 76 eligible articles validated 34 vascular surgery simulators and training courses for open and endovascular procedures. High validity ratings were achieved across studies for: content , response processes , the internal structure , relations to other variables, and consequences. Only seven studies achieved an LoE greater than 3/5. Overall, ANGIO Mentor was the most highly validated and effective simulator and was the only simulator to achieve an LoE of 5/5.

Conclusions: Simulation-based training in vascular surgery is a continuously developing field with exciting future prospects, demonstrated by the vast number of models and training courses. To effectively integrate simulation models into current vascular surgery curricula and assessments, there is a need for studies to look at trainee skill retention over a longer period of time. A more detailed discussion on cost-effectiveness is also needed.

Objectives: Healthcare evolution requires optimisation of surgical training to provide safe patient care. Operating
room performance after completion of proficiency based training in vascular surgery has not been investigated.

Design: A randomised controlled trial evaluated the impact of a Proficiency based Stepwise Endovascular
Curricular Training program (PROSPECT) on the acquisition of endovascular skills and the transferability of these
skills to real life interventions.

Materials: All subjects performed two endovascular interventions treating patients with symptomatic iliac and/or
superficial femoral artery stenosis under supervision. Primary outcomes were technical performances (Global Rating
Scale [GRS]; Examiner Checklist), operative metrics, and patient outcomes, adjusted for case difficulty and trainee
experience. Secondary outcomes included knowledge and technical performance after 6 weeks and 3 months.

Methods: Thirty-two general surgical trainees were randomised into three groups. Besides traditional training, the first group (n ¼ 11) received e-learning and simulation training (PROSPECT), the second group (n ¼ 10) only had access to e-learning, while controls (n ¼ 11) did not receive supplementary training.

Results: Twenty-nine trainees (3 dropouts) performed 58 procedures. Trainees who completed PROSPECT showed
superior technical performance (GRS 39.36 +/- 2.05; Checklist 63.51 +/- 3.18) in real life with significantly fewer
supervisor takeovers compared with trainees receiving e-learning alone (GRS 28.42 +/- 2.15; p ¼ .001; Checklist
53.63 +/- 3.34; p ¼ .027) or traditional education (GRS 23.09 +/- 2.18; p ¼ .001; Checklist 38.72 +/- 3.38; p ¼ .001).
Supervisors felt more confident in allowing PROSPECT trained physicians to perform basic (p ¼ .006) and complex
(p ¼ .003) procedures. No differences were detected in procedural parameters (such as fluoroscopy time, DAP,
procedure time, etc.) or complications. Proficiency levels were maintained up to 3 months.

Conclusions: A structured, stepwise, proficiency-based endovascular curriculum including e-learning and
simulation-based training should be integrated early into training programs to enhance trainee performance.

Introduction: Simulator training can potentially provide high-intensity training in electrophysiology (EP) without compromising patient safety. We assessed the construct validity of a novel EP simulator (ANGIO Mentor; Simbionix) and developed proficiency-based scores for clinical EP simulator training.

Methods: Two European training centers participated. Participants were grouped on the basis of years of EP experience and (for a subset) subjectively scored clinical catheter manipulation skills. Each participant attempted the simulator’s 5 modules 3 times. These modules focus on catheter manipulation in 3-dimensional models, ranging from geometric shapes to fluoroscopic contracting cardiac models, with performance scored by the system on the basis of attainment of preset targets. Using these scores, targets were formulated for basic EP training.

Results: Twenty-eight participants were included (13 with subjectively scored catheter manipulation). Scores for participants with less than 1-year EP experience (group 1) were significantly lower for each of the attempts at the modules (P < 0.002). For group 1 only, scores improved with subsequent attempts (P < 0.005). In 4 of the 5 modules, scores of group 1 were significantly lower than the more experienced groups (P < 0.0005). Participants with subjectively scored above-average catheter manipulation skills also had higher scores in 4 of the 5 modules (P < 0.05). Target scores for a proficiency-based training program were generated from the median scores for each module for those with 1-year experience or more.

Conclusions: Scores attained in the simulator can distinguish those with less than 1-year EP experience and those with above-average catheter manipulation skills. Consequently, target scores have been generated for a proficiency-based training program.

Background: Realistic virtual reality (VR) simulators have greatly expanded the tools available for training surgeons and interventionalists. While this technology is effective in improving performance in many fields, it has never been evaluated for neuroendovascular procedures. This study aims to determine whether VR is an effective tool for improving neuroendovascular skill among trainees.

Methods: Trainees performed two VR revascularizations of a right-sided middle cerebral artery (MCA) thrombosis and their times to procedural benchmarks (time to enter internal carotid artery [ICA], traverse clot, and complete procedure) were compared. To determine whether the improvement was case specific, trainees with less procedural exposure were timed during VR left-sided ICA (LICA) aneurysm coiling before or after performing MCA thrombectomy simulations. To determine the value of observing simulations, medical students were timed during the right MCA revascularization simulations after watching other VR procedures.

Results: Trainees significantly improved their time to every procedural benchmark during their second MCA revascularization (mean decrease = 1.08, 1.57, and 2.24 min; P = 0.0072, 0.0466, and 0.0230). In addition, time required to access the LICA during aneurysm coiling was shortened by 0.77 min for each previous VR right MCA revascularization performed (P = 0.0176; r² = 0.71). Finally, medical students’ MCA revascularization simulation times improved by 0.87 min for each prior simulation viewed (P r² = 0.96).

Conclusion: Both performance and viewing of simulated procedures produced significant decreases in time to reach neuroendovascular procedural benchmarks. These data show that VR simulation is a valuable tool for improving trainee skill in neuroendovascular procedures.

Objective and Background: This study aimed to determine the cost-effectiveness of a Proficiency based Stepwise Endovascular Curricular Training (PROSPECT) program, including e-learning and hands on virtual reality simulation.

Methods: A prospective, single blinded, randomised controlled trial (RCT) was carried out to evaluate the impact of a PROSPECT training program on real life operative performance. Under supervision, all subjects performed two endovascular interventions on patients with symptomatic iliac and/or superficial femoral artery stenosis. Primary outcomes were technical performance (Global Rating Scale, Examiner Checklist), operative metrics, and patient outcomes, adjusted for case difficulty and the trainee’s experience. Additionally, an analysis of costs and savings related to implementation of this endovascular training program was performed. Thirty-two general surgery trainees were randomised into three groups: group 1 (n = 11) received e-learning and simulation training (PROSPECT program); group 2 (n = 10) only had access to e-learning; group 3 (n = 11) did not receive supplementary education besides clinical training. Developmental cost, implementation cost, training time cost, and the operational cost of PROSPECT were determined. Time spent studying and practicing was converted to indirect saving of operating time. The costs of logistics, faculty time supervising simulation sessions, and 30 day complication rates were registered. Sensitivity analysis was performed to assess the robustness of the results.

Results: Fifty-eight peripheral endovascular interventions, performed by 29 surgical trainees (three dropouts) were included in this RCT from October 2014 to February 2016. Annual costs from the perspective of the hospital were €6589 for curriculum design, €31,484 for implementation, and €1143 in operational costs. Per trainee, simulation-based training until proficiency cost €3806. In comparison, if endovascular proficiency levels were obtained with conventional training only, this may have cost €5001 per trainee.

Conclusion: Simulation based training in endovascular procedures may be cost saving, because training occurs outside the angiosuite. It is possible that cost savings are underestimated as, in contrast to the literature, prevented costs related to complications could not be defined.