Significance of Robotic VR Simulation on OR Performance

As robotic surgeons, we have all used virtual reality (VR) robotic surgery simulators; although very few of us have had to utilize these in required training curriculums that mandate achieving proficiency prior to starting to operate on live patients.

A recently published study in the British Journal of Surgery out of Heidelberg University in Germany did a very extensive review of all studies discussing VR simulation training trying to identify studies that actually evaluated the concurrent validity (CV) and predictive validity (PV) of virtual reality (VR) training for robotic surgery.¹  See validation study here

This study conducted an extensive systematic review and meta-analysis of  all published robotic VR simulation studies seeking to identify those studies which discussed the evidence on transferability of surgical skills that were acquired on robotic VR simulators to the OR and also to identify those studies that identified the predictive value of robotic VR simulation training that transferred into the OR. The authors identified 14,131 potential articles on robotic VR simulation training, but only eight of these studies qualified for their qualitative analysis and only three qualified for their quantitative analysis.

There is an abundance of literature supporting skill transfer from laparoscopic VR simulators to the OR,^2,3 but there is very little evidence of skills transfer to the OR from Robotic VR Simulators. The authors did an extensive literature review utilizing high-level statistical analyses to narrow down and identify the few studies that qualified for this very vigorous review. The outcomes they decided on included not only objective metrics (ROSATS, GEARS, simulator metrics, etc.) but also operative outcomes including time needed for exercises and/or surgical procedures, estimated blood loss, conversion rates, and finally patient related outcomes such as length of stay and post-operative pain. There were five studies that demonstrated skills transfer (CV) to the actual robot and three additional studies that confirmed predictability of skills transfer (PV) to surgical outcomes. The analysis was exhaustive and involved very sophisticated statistical analytics to rule out risk of bias and to determine the degree of correlation among similar types of studies.

The good news is that their findings did indeed show positive correlation with VR skills training and measured OR outcomes. In addition, they also identified one study that showed a significant reduced workload for the student surgeons citing decreased mental demand, physical demand, temporal demand, effort, frustration and increased performance on the daVinci surgical console after robotic VR simulation training. The conclusion of the authors was that their review did confirm that certain surgical skills and improved technical performance acquired on robotic VR simulators can be transferred to the OR (CV & PV), although they state that the extent to which this benefit can be applied “still remains unclear.”

One confounding issue in comparing these studies is the variability in the VR curriculums being used: are they just curriculums that need to be passed once or do they indeed train to proficiency, typically defined as at least two consecutive passing scores on each exercise?⁴  Another confounding areas is improved OR time after VR simulation training. If surgical time improvement was the most important measure of validity, the outcome of shortened OR times for new surgeons would clearly justify the cost of VR simulators in training based on the high cost/minute of OR time; however, a recent Delphi survey of laparoscopic surgery experts reported by these same authors considered OR time to be the least important indicator of OR surgical performance on simulators compared with safety, dexterity and efficiency.⁵

My take-aways from this study is that the focus of future research needs to consider both skills transfer (Concurrent validity studies) as well as improvement in intraoperative performance and outcomes (Predictive validity studies).  That is challenging because of the complex nature of OR procedures themselves which, as we all recognize, have great variability and are therefore much more difficult to create standardize scoring for similar to how we score simulation tasks.  All that we can say now for sure is that performance on robotic VR simulators seems to predict a higher level of technical performance in the OR. Further research will need to be done on large populations of surgeons and patients, likely using machine learning algorithms, to really tease out the benefits of universal robotic VR simulation training has on overall cost, safety and patient outcomes. This will almost certainly be the way we train and evaluate surgeons in the future and bring us into line with other highly reliable industries such as where safety and good outcomes are always the most critical consideration.

Author:

John Lenihan Jr., M.D.