Curricula

Assessing the Effectiveness of Surgical Skills Laboratories.

Varban OA et al.

Sim Healthcare 2013, vol 8, p91-97

INTRODUCTION: Surgical Skills laboratories have gained widespread use in surgery residency training. Although the availability of simulators and skills laboratories has expanded, little is known about their use and effect on residency training.

METHODS: an online survey consisting of 18 questions was distributed to all members of the Association of Program Directors in Surgery. The survey addressed surgical skills laboratory funding, types of simulators, curricula, tools for evaluation, and opinions on successful implementation of a skills laboratory. Statistical analysis was performed on data obtained from completed surveys.

RESULTS: a total of 248 programs were invited to respond to the survey, and 81 responses were obtained (33% response rate). Among programs that responded, most mandate time for residents to use their skills laboratory (76%), and most offer a formal curriculum (63%). Few programs require demonstrated proficiency before participating in the operating room (16%), and only 55% of responders believed that their resident’s overall intraoperative technical skills had improved since the implementation of their skills laboratory. Respondents believed that interns derive the most benefit from their skills laboratory when compared with all other years of surgical training (P<0.001). Faculty participation was most commonly considered as the most important factor for successful implementation of a surgical skills laboratory, regardless of program characteristics.

CONCLUSION: Among surgical residency programs that responded to the survey, most programs schedule time for residents to use a skills laboratory, and most use a curriculum. Those surveyed report that interns derive the most benefit, whereas chief residents derive the least. They also believe that faculty participation is the most important aspect to successful implementation of a skills laboratory.

A Comparison of 2 Ex Vivo Training Curricula for Advanced Laparoscopic Skills: A Randomized Controlled Trial

Orzech N, Palter VN, Reznick RK, Aggarwal R, Grantcharov TP

Ann Surg 2012, Vol 255(5), p833–839

OBJECTIVE: To compare the effectiveness and cost of 2 ex vivo training curricula for laparoscopic suturing. BACKGROUND: Although simulators have been developed to teach laparoscopic suturing, a barrier to their wide implementation in training programs is a lack of knowledge regarding their relative training benefit and their associated cost.

METHOD: This prospective single-blinded randomized trial allocated 24 surgical residents to train to proficiency using either a virtual reality (VR) simulator or box trainer. All residents then placed intracorporeal laparoscopic stitches during a Nissen fundoplication on a patient. The operating room (OR) cases were video-recorded and technical proficiency was assessed using 2 validated tools. OR performance of both groups was compared to that of conventionally trained residents and to fellowship-trained surgeons. A cost analysis of box training, VR training, and conventional residency training across Canadian surgical programs was performed.

RESULTS: After ex vivo training, no significant differences in laparoscopic suturing in the OR were found between the 2 groups with respect to time (P = 0.74)—global rating score (P = 0.65) or checklist score (P = 0.97). It took conventionally trained residents 6 practice attempts in the OR to achieve the technical proficiency of the ex vivo trained groups (P = 0.83). VR training was more efficient than box training (transfer effectiveness ratio of 2.31 vs 1.13). The annual cost of training 5 residents on the FLS trainer box was $11,975.00, on the VR simulator was $77,500.00, and conventional residency training was $17,380.00. Over 5 years, box training was the most cost-effective option for all programs, and VR training was more cost-effective for programs with more 10 residents.

CONCLUSIONS: Training on either a VR simulator or on a box trainer significantly decreased the learning curve necessary to learn laparoscopic suturing. VR training, however, is the more efficient training modality, whereas box training the more cost-effective option.

Designing a proficiency-based, content validated virtual reality curriculum for laparoscopic colorectal surgery: A Delphi approach

Palter VN, Graafland M, Schijven MP, Grantcharov TP,

Surgery 2012 vol 151, p391-7

BACKGROUND: Although task training on virtual reality (VR) simulators has been shown to transfer to the operating room, to date no VR curricula have been described for advanced laparoscopic procedures. The purpose of this study was to develop a proficiency-based VR technical skills curriculum for laparoscopic colorectal surgery.

METHODS: The Delphi method was used to determine expert consensus on which VR tasks (on the LapSim simulator) are relevant to teaching laparoscopic colorectal surgery. To accomplish this task, 19 international experts rated all the LapSim tasks on a Likert scale (1–5) with respect to the degree to which they thought that a particular task should be included in a final technical skills curriculum. Results of the survey were sent back to participants until consensus (Cronbach’s α >0.8) was reached. A cross-sectional design was utilized to define the benchmark scores for the identified tasks. Nine expert surgeons completed all identified tasks on the ‘‘easy’’, ‘‘medium,’’ and ‘‘hard’’ settings of the simulator.

RESULTS: In the first round of the survey, Cronbach’s α was 0.715; after the second round, consensus was reached at 0.865. Consensus was reached for 7 basic tasks and 1 advanced suturing task. Median expert time and economy of movement scores were defined as benchmarks for all curricular tasks.

CONCLUSION: This study used Delphi consensus methodology to create a curriculum for an advanced laparoscopic procedure that is reflective of current clinical practice on an international level and conforms to current educational standards of proficiency-based training.

Development and Validation of a Comprehensive Curriculum to Teach an Advanced Minimally Invasive Procedure – A Randomized Controlled Trial.

Palter VN and Grantcharov TP.

Ann Surg 2012 (256) 1219–1226.

OBJECTIVE: This study allocated 25 surgical residents to receive either conventional residency training or a comprehensive training curriculum for laparoscopic colorectal surgery. All participants performed a laparoscopic right colectomy, which was video recorded and assessed using 2 previously validated assessment tools. Secondary outcome measures were knowledge relating to the execution of the procedure, assessed with a multiple-choice test, and technical performance on the simulator.

RESULT: Curricular-trained residents demonstrated superior performance in the operating room compared with conventionally trained residents. Curricular-trained residents scored higher on the multiple-choice test and outperformed conventionally trained residents in 7 of 8 tasks on the simulator.

CONCLUSION: Participation in a comprehensive ex vivo training curriculum for laparoscopic colorectal surgery results in improved technical knowledge and improved performance in the operating room compared with conventional residency training.

Validation of a Structured Training and Assessment Curriculum for Technical Skill Acquisition in Minimally Invasive Surgery: A Randomized Controlled Trial

Palter VN, Orzech N, Reznick RK, Grantcharov T

Ann Surg 2013 Volume 257(2), p224–230

OBJECTIVE: To develop and validate an ex vivo comprehensive curriculum for a basic laparoscopic procedure.

BACKGROUND: Although simulators have been well validated as tools to teach technical skills, their integration into comprehensive curricula is lacking. Moreover, neither the effect of ex vivo training on learning curves in the operating room (OR), nor the effect on nontechnical proficiency has been investigated.

METHODS: This randomized single-blinded prospective trial allocated 20 surgical trainees to a structured training and assessment curriculum (STAC) group or conventional residency training. The STAC consisted of case-based learning, proficiency-based virtual reality training, laparoscopic box training, and OR participation. After completion of the intervention, all participants performed 5 sequential laparoscopic cholecystectomies in the OR. The primary outcome measure was the difference in technical performance between the 2 groups during the first laparoscopic cholecystectomy. Secondary outcome measures included differences with respect to learning curves in the OR, technical proficiency of each sequential laparoscopic cholecystectomy, and nontechnical skills.

RESULTS: Residents in the STAC group outperformed residents in the conventional group in the first (P = 0.004), second (P = 0.036), third (P = 0.021), and fourth (P = 0.023) laparoscopic cholecystectomies. The conventional group demonstrated a significant learning curve in the OR (P = 0.015) in contrast to the STAC group (P = 0.032). Residents in the STAC group also had significantly higher nontechnical skills (P = 0.027).

CONCLUSIONS: Participating in the STAC shifted the learning curve for a basic laparoscopic procedure from the operating room into the simulation laboratory. STAC-trained residents had superior technical proficiency in the OR and nontechnical skills compared with conventionally trained residents. (The study registration ID is NCT01560494.)

Instructor Feedback Versus No Instructor Feedback on Performance in a Laparoscopic Virtual Reality Simulator: A Randomized Trial

Strandbygaard J, Bjerrum F, Maagaard M, Winkel P, Larsen CR, Ringsted C, Gluud C, Grantcharov T, Ottesen B, Sorensen JL
Ann Surg 2013, Volume 257(5) p839–844

OBJECTIVE: To investigate the impact of instructor feedback versus no instructor feedback when training a complex operational task on a laparoscopic virtual reality simulator.
BACKGROUND: Simulators are now widely accepted as a training tool, but there is insufficient knowledge about how much feedback is necessary, which is useful for sustainable implementation.
METHODS: A randomized trial complying with CONSORT Statement. All participants had to reach a predefined proficiency level for a complex operational task on a virtual reality simulator. The intervention group received standardized instructor feedback a maximum of 3 times. The control group did not receive instructor feedback. Participants were senior medical students without prior laparoscopic experience (n = 99). Outcome measures were time, repetitions, and performance score to reach a predefined proficiency level. Furthermore, influence of sex and perception of own surgical skills were examined.
RESULTS: Time (in minutes) and repetitions were reduced in the intervention group (162 vs 342 minutes; P < 0.005) and (29 vs 65 repetitions; P < 0.005). The control group achieved a higher performance score than the intervention group (57% vs 49%; P = 0.004). Men used less time (in minutes) than women (P = 0.037), but no sex difference was observed for repetitions (P = 0.20). Participants in the intervention group had higher self-perception regarding surgical skills after the trial (P = 0.011).
CONCLUSIONS: Instructor feedback increases the efficiency when training a complex operational task on a virtual reality simulator; time and repetitions used to achieve a predefined proficiency level were significantly reduced in the group that received instructor feedback compared with the control group.

Simulation in surgical education

Vanessa N. Palter MD, Teodor P. Grantcharov MD PhD.
Surg Clin N Am 90 (2010) 605–617

With recent concerns regarding patient safety, and legislation regarding resident work hours, it is accepted that a certain amount of surgical skills training will transition to the surgical skills laboratory. Virtual reality offers enormous potential to enhance technical and non-technical skills training outside the operating room. Virtual-reality systems range from basic low-fidelity devices to highly complex virtual environments. These systems can act as training and assessment tools, with the learned skills effectively transferring to an analogous clinical situation. Recent developments include expanding the role of virtual reality to allow for holistic, multidisciplinary team training in simulated operating rooms, and focusing on the role of virtual reality in evidence-based surgical curriculum design.

What is the ideal interval between training sessions during proficiency-based laparoscopic simulator training?

Stefanidis D., Walters C, Mostafavia A, B. Heniford T

The American Journal of Surgery Volume 197, Issue 1, January 2009, Pages 126-129

BACKGROUND: The aim of this study was to identify the ideal interval between training sessions in a proficiency-based laparoscopic suturing simulator curriculum. METHODS: We analyzed performance data from 3 randomized controlled trials of novices (n = 66) who followed a similar proficiency-based simulator curriculum in laparoscopic suturing on the Fundamentals of Laparoscopic Surgery model. The change in performance and intertraining interval were correlated.

RESULTS: Overall participant performance improved from 530 +/- 58 seconds at baseline to 81 +/- 14 seconds at training completion (P < .001). Intertraining intervals ranged from 1 to 43 days and performance change between training sessions varied widely. There was no correlation of performance change with intertraining interval (r = .05, P = .30). Performance deterioration was similar at different intertraining intervals. Shorter intervals were associated, however, with shorter training duration (r = .35, P = .005).

CONCLUSIONS: No association was found between intertraining interval and change in performance during proficiency-based laparoscopic simulator training but shorter intervals were associated with improved skill acquisition. Further study is needed to confirm these findings.

Designing and validating a customized virtual reality-based laparoscopic skills curriculum.

Panait L, Bell RL, Roberts KE, Duffy AJ.

Surg Educ. 2008 Nov-Dec;65(6):413-7.

OBJECTIVE: We developed and instituted a laparoscopic skills curriculum based on a virtual reality simulator, LapSim (Surgical Science, Göteborg, Sweden). Our goal was to improve basic skills in our residents. The hypothesis of this study is that performance in our course will differentiate levels of experience in the training program, establishing construct validity for our curriculum. DESIGN: We designed a novel curriculum that consisted of 17 practice modules and a 7-part examination. All residents who completed the curriculum successfully were included in this study. Performance to complete the examination was analyzed. Data were stratified by level of training. SETTING: University surgical skill training laboratory. PARTICIPANTS: In all, 29 residents of all levels of training and 3 attending surgeons completed the curriculum.

RESULTS: The average number of practice repetitions required was 243. To complete the examination component, junior residents (R1-R3) required more repetitions than senior residents (R4, R5), 28.3 versus 13.9, respectively (p < 0.002). Tasks on camera and instrument navigation as well as coordination did not reveal significant differences. The complex grasping task demonstrated significant differences in repetitions required for each level of training: 19.5 attempts for R1, 17.2 for R2, 13 for R3, 8.5 for R4, and 3 for R5 (p < 0.04). The 2 cutting drills, which required precise use of the left hand, required 7.9 repetitions for junior residents versus 2.7 for senior residents (p < 0.009). A clip application drill differentiated among junior residents with 39.4, 19.8, and 8.5 repetitions required for R1, R2, and R3, respectively (p < 0.05). Senior residents performed equivalent to attendings on this drill. A lifting and grasping drill differentiates among junior residents, senior residents, and attendings (p < 0.03).

CONCLUSIONS: Individual performance in our curriculum correlates with the level of training for many drills, which establishes construct validity for this curriculum. Noncontributory drills may need to be revised or removed from the curriculum. Successful completion of this curriculum may lead to improved resident technical performance.

The future of patient safety: Surgical trainees accept virtual reality as a new training tool.

Rosenthal R, Gantert WA, Hamel C, Metzger J, Kocher T, Vogelbach P, Demartines N, Hahnloser D.

Patient Saf Surg. 2008 Jun 11;2:16.C

ABSTRACT: BACKGROUND: The use of virtual reality (VR) has gained increasing interest to acquire laparoscopic skills outside the operating theatre and thus increasing patients’ safety. The aim of this study was to evaluate trainees’ acceptance of VR for assessment and training during a skills course and at their institution. METHODS: All 735 surgical trainees of the International Gastrointestinal Surgery Workshop 2006-2008, held in Davos, Switzerland, were given a minimum of 45 minutes for VR training during the course. Participants’ opinion on VR was analyzed with a standardized questionnaire.

RESULTS: Fivehundred-twenty-seven participants (72%) from 28 countries attended the VR sessions and answered the questionnaires. The possibility of using VR at the course was estimated as excellent or good in 68%, useful in 21%, reasonable in 9% and unsuitable or useless in 2%. If such VR simulators were available at their institution, most course participants would train at least one hour per week (46%), two or more hours (42%) and only 12% wouldn’t use VR. Similarly, 63% of the participants would accept to operate on patients only after VR training and 55% to have VR as part of their assessment.

CONCLUSION: Residents accept and appreciate VR simulation for surgical assessment and training. The majority of the trainees are motivated to regularly spend time for VR training if accessible.

Documenting a learning curve and test-retest reliability of two tasks on a virtual reality training simulator in laparoscopic surgery.

Hogle NJ, Briggs WM, Fowler DL.

J Surg Educ. 2007 Nov-Dec;64(6):424-30

BACKGROUND: Virtual reality simulators are a component of the armamentarium for training surgical residents. No one knows exactly how to incorporate virtual reality simulators into a curriculum. The purpose of this study was to document and show the learning curve and test-retest reliability of 2 tasks on a virtual reality-training simulator (LapSim; Surgical Science, Göteborg, Sweden) in laparoscopic surgery.

METHODS: Twenty-nine medical students participated in 8 iterations of 7 virtual reality tasks (“camera navigation” (CN), “instrument navigation,” “coordination,” “grasping,” “lifting and grasping” (LG), “cutting,” and “clip applying”) Learning curves for each outcome variable of the CN and LG tasks were generated. Using ANOVA, we evaluated the differences between each score from attempt number 7 to attempt number 8 to document test-retest reliability.

RESULTS: A plateau in the learning curve occurred within 8 sessions for CN misses, CN tissue damage, CN maximum damage, and LG maximum damage. Over the course of 8 sessions, a plateau in the learning curve was nearly reached for CN time, CN drift, CN path, CN angular path, and LG left and right path. The following variables had a downward trend to the mean learning curve over 8 sessions, but they did not reach a plateau: LG time, LG left and right miss, LG left and right angular path, and LG tissue damage.

CONCLUSION: Using the LapSim virtual reality simulator, we documented a learning curve and test-retest reliability for each outcome variable for CN and LG for rank novices. The modeling of the general learning curve is useful in designing training program. These results may be important in developing standards for technical evaluation in a surgical training curriculum.

Criterion-based training with surgical simulators: proficiency of experienced surgeons.

Heinrichs WL, Lukoff B, Youngblood P, Dev P, Shavelson R, Hasson HM, Satava RM, McDougall EM, Wetter PA.

JSLS. 2007 Jul-Sep;11(3):273-302.

OBJECTIVE: In our effort to establish criterion-based skills training for surgeons, we assessed the performance of 17 experienced laparoscopic surgeons on basic technical surgical skills recorded electronically in 26 modules selected in 5 commercially available, computer-based simulators.

METHODS: Performance data were derived from selected surgeons randomly assigned to simulator stations, and practicing repetitively during one and one-half day sessions on 5 different simulators. We measured surgeon proficiency defined as efficient, error-free performance and developed proficiency score formulas for each module. Demographic and opinion data were also collected.

RESULTS: Surgeons’ performance demonstrated a sharp learning curve with the most performance improvement seen in early practice attempts. Median scores and performance levels at the 10th, 25th, 75th, and 90th percentiles are provided for each module. Construct validity was examined for 2 modules by comparing experienced surgeons’ performance with that of a convenience sample of less-experienced surgeons.

CONCLUSION: A simple mathematical method for scoring performance is applicable to these simulators. Proficiency levels for training courses can now be specified objectively by residency directors and by professional organizations for different levels of training or post-training assessment of technical performance. But data users should be cautious due to the small sample size in this study and the need for further study into the reliability and validity of the use of surgical simulators as assessment tools.

An evidence-based virtual reality training program for novice laparoscopic surgeons.

Aggarwal R, Grantcharov TP, Eriksen JR, Blirup D, Kristiansen VB, Funch-Jensen P, Darzi A.

Ann Surg. 2006 Aug;244(2):310-4.

OBJECTIVE: To develop an evidence-based virtual reality laparoscopic training curriculum for novice laparoscopic surgeons to achieve a proficient level of skill prior to participating in live cases. SUMMARY BACKGROUND DATA: Technical skills for laparoscopic surgery must be acquired within a competency-based curriculum that begins in the surgical skills laboratory. Implementation of this program necessitates the definition of the validity, learning curves and proficiency criteria on the training tool.

METHODS: The study recruited 40 surgeons, classified into experienced (performed >100 laparoscopic cholecystectomies) or novice groups (<10 laparoscopic cholecystectomies). Ten novices and 10 experienced surgeons were tested on basic tasks, and 11 novices and 9 experienced surgeons on a procedural module for dissection of Calot triangle. Performance of the 2 groups was assessed using time, error, and economy of movement parameters.

RESULTS: All basic tasks demonstrated construct validity (Mann-Whitney U test, P < 0.05), and learning curves for novices plateaued at a median of 7 repetitions (Friedman’s test, P < 0.05). Expert surgeons demonstrated a learning rate at a median of 2 repetitions (P < 0.05). Performance on the dissection module demonstrated significant differences between experts and novices (P < 0.002); learning curves for novice subjects plateaued at the fourth repetition (P < 0.05). Expert benchmark criteria were defined for validated parameters on each task.

CONCLUSION: A competency-based training curriculum for novice laparoscopic surgeons has been defined. This can serve to ensure that junior trainees have acquired prerequisite levels of skill prior to entering the operating room, and put them directly into practice.

Development of a valid, cost-effective laparoscopic training program.

Adrales GL, Chu UB, Hoskins JD, Witzke DB, Park AE

Am J Surg. 2004 Feb;187(2):157-63

BACKGROUND: Practical programs for training and evaluating surgeons in laparoscopy are needed to keep pace with demand for minimally invasive surgery.

METHODS: At the University of Kentucky five inexpensive simulations have been developed to train and assess surgical residents. Residents are videotaped performing laparoscopic procedures on models. Five surgeons assess the taped performances on 4 global skills.

RESULTS: Creating mechanical models reduces training costs. Trainees agreed procedures were well represented by the simulations. Blinded assessment of performances showed high interrater agreement and correlated with the trainees’ level of experience. Nonclinician evaluations on checklists correlated with evaluations by surgeons.

CONCLUSIONS: Inexpensive simulations of laparoscopic appendectomy, cholecystectomy, inguinal herniorrhaphy, bowel enterotomy, and splenectomy enable surgical residents to practice laparoscopic skills safely. Obtaining masked, objective, and independent evaluations of basic skills in laparoscopic surgery can assist in reliable assessment of surgical trainees. The simulations described can anchor an innovative educational program during residency for training and assessment.

Completion of a novel, virtual-reality-based, advanced laparoscopic curriculum improves advanced laparoscopic skills in senior residents.

Panait L, Hogle NJ, Fowler DL, Bell RL, Roberts KE, Duffy AJ.
J Surg Educ. 2011 Mar-Apr;68(2), p121-5.

INTRODUCTION: Virtual reality simulators contribute to basic laparoscopic skill acquisition. These trainers have not yet been shown to contribute to the acquisition of more advanced laparoscopic skills as measured by the Fundamentals of Laparoscopic Surgery (FLS). We have customized novel basic and advanced curricula for the LapSim trainer (Surgical Science, Göteborg, Sweden). Successful completion of these programs is required of our residents. We hypothesize that the successful completion of our advanced curriculum will result in the significant improvement of our residents’ advanced laparoscopic skills as measured by the FLS skills scores.
METHODS: In all, 23 surgical residents (PGY 1-4), who had already passed our basic skills curriculum, completed our advanced LapSim curriculum. All individuals underwent FLS skills testing before and after completing the training. Laparoscopic case experience during the training period was documented for all trainees. FLS scores were analyzed by t test and controlled for case experience.
RESULTS: Posttraining FLS scores demonstrate a significant increase for all residents from a mean of 57-66 (p < 0.02), especially for seniors (PGY 3-4): 56-68 (p < 0.01). The operative laparoscopic case volume ranged from 1-90 (mean, 30) for juniors (PGY 1-2) and 12-76 (mean 50) for seniors during the training period. Junior resident FLS improvement was dependent on case volume during the period of training; residents with less than 30 cases had a mean improvement of 0, whereas those with at least 30 cases had a 15 point improvement (p < 0.01). Senior resident FLS score improvement was independent of case numbers during the training period.
CONCLUSIONS: Completion of our advanced LapSim curriculum results in improved advanced laparoscopic skills in senior residents as measured by FLS scores. This skill improvement is independent of laparoscopic case experience. Continuing to mandate the use of this skills curriculum should improve our residents’ performance in advanced laparoscopic surgical procedures.