Study Protocol Title: Evaluation of the Fundamentals of Robotic Surgery (FRS) Online and Psychomotor Skill Curriculum: Validation Pilot Study

Transcription

1 Study Protocol Title: Evaluation of the Fundamentals of Robotic Surgery (FRS) Online and Psychomotor Skill Curriculum: Validation Pilot Study Principal Investigator: Roger Smith, PhD Sub Investigator: Manuella Perez, MD Site Study Address: Florida Hospital Nicholson Center 404 Celebration Place Celebration, Florida Previous Version Date: Page 1 of 52

2 Table of contents List of Abbreviations:...4 Introduction...5 Background Information and Scientific Rationale...5 Study Objectives...9 Primary Objective/Aim/Goal/Hypothesis... 9 Secondary Objective/Aim/Goal/Hypothesis... 9 Study Design...10 Research Design Study Agent, Device, and/or Intervention Description Study Site(s)/Location(s) and Number of Subjects Multi-Site Research Logistics/Communication Plan Community-Based Participatory Research Subject Selection...21 Inclusion Criteria Exclusion Criteria Study Procedures...21 Site Initiation/Study-Specific Training Subject Recruitment and Screening Consent Procedures Waiver of Written Documentation of Consent or Waiver of Consent Documentation of Informed Consent Process Randomization Study Visits Study Duration Materials of Human Origin: Collection, Preparation, Handling and Shipping Study Outcome Measures (Endpoints)...23 Data Management and Quality Plan...23 Data De-identification Data Confidentiality, Storage, and Retention Data Quality Data Sharing (outside of Florida Hospital) Sample Size Determination...25 Statistical Analysis Plan...25 Primary Objective Analysis Secondary Objective Analysis Potential Risks and Benefits...26 Potential Benefits Potential Risks Potential Data Risks Provisions to Protect the Privacy Interest of Subjects Previous Version Date: Page 2 of 52

3 Early Withdrawal of Subjects...26 Investigator Withdrawal of Subjects Subject Request for Withdrawal from Study Data Collection and Follow-up for Withdrawn Subjects Adverse Event Reporting:...27 Adverse Events Recording of Adverse Events Notification of Adverse Events Safety Monitoring Plan...27 Safety Monitoring Data and Safety Monitoring Board (DSMB) or Equivalent Ethical Considerations...27 Sharing of Results with Subjects Funding Source...27 Conflict of Interest Subject Stipends or Payments...27 Publication Plan...28 S: REFERENCES...29 APPENDIX...30 Previous Version Date: Page 3 of 52

4 List of Abbreviations: FLS: Fundamentals of Laparoscopic Surgery FRS: Fundamentals of Robotic Surgery GEARS: Global Evaluation Assessment of Robotic Skills SAGES: Society of American Gastrointestinal and Endoscopic Surgeons MIRA: Minimally Invasive Robotics Association SRS: Society of Robotic Surgery DVSS: Da Vinci Skills Simulator DVT: dv-trainer DoD: Department of Defense Appendix Index: There are 6 appendixes provided in this protocol. Appendix 1: Demographic Data (page 31) To collect demographic data concerning the experts and novices Appendix 2: Checklist for assessment of the FRS and Intuitive Surgical online curriculums (page 32-36) To assess the retention of information after taking the FRS or Intuitive Surgical Curriculum and to test the transfer of knowledge from the e-learning to the real robot management. This assessment will be administered to novices only. Appendix 3: Questionnaire for FRS Psychomotor Skill Curriculum Validation (page 37-41) To evaluate the accuracy of the seven psychomotor tasks in addressing the skills which they were designed to test. Appendix 4: Questionnaire to evaluate the Effectiveness of FRS Curriculum To subjectively evaluate the Effectiveness of FRS Curriculum according to the 25 fundamental skills representing the tasks performed in robotic surgery. The questionnaire will be completed only by the expert group. Appendix 5: Global Evaluative Assessment of Robotic Skills (GEARS) The GEARS scale is a validated assessment tool comprised of various metrics to evaluate skills in robotic surgery. This will be used to evaluate the subjects performance on the psychomotor skills tasks via video recordings. Appendix 6: Standardized explanation of the OR process and equipment for the psychomotor skills evaluation (page 46-54). To provide a standardized explanation of the Psychomotor Skills Exercises and Metrics to both types of subjects: experts and novices. Previous Version Date: Page 4 of 52

5 Introduction This document is a protocol for a human randomized prospective research study. This study is to be conducted in accordance with applicable Federal regulations and institutional research policies and procedures. The aim of this study is to evaluate the process, which will be used for the validation of the Fundamentals of Robotic Surgery (FRS) Online and Psychomotor Skills Curriculum. This study is not itself, the validation study, rather it is a pilot study designed to evaluate all of the processes that will be used in a much larger multi-site validation trial which will include up to 10 different institutions and hundreds of subjects. This pilot study will be conducted entirely within the Florida Hospital system and the knowledge gained will be used to design the experiment and protocol for the much larger study. Background Information and Scientific Rationale Exponential growth of minimally invasive procedures, especially robotic-assisted procedures, raises the question of how to assess robotic surgery skills. Robotic surgery has been established as an innovative approach in surgery thanks to a telemanipulator device which introduced a new dimension to the surgical tool box. Robotic surgical systems overcome the laparoscopic limitations and facilitate the performance of minimally invasive surgery in complex procedures due to better 3D vision, 7-degree-of-freedom instruments, tremor abolition, motion amplification, and stable camera. However, this new device also introduces a specific need for training and certification to ensure a minimal standard of care for all patients undergoing robotic surgery. Surgical Societies (e.g. SAGES, SRS, and MIRA) are encouraging the development of a unified approach and standard for basic training in robotic surgery skills principles (1). Currently, surgery and medicine have come to the conclusion that the Halstedian training paradigm, developed in the 19 century, is no longer optimal for the acquisition of these types of procedural skills, due to the cost and the complexity of the procedure performed in robotic surgery. In the early 1990s, Dr. Satava proposed virtual reality (VR) simulation training for the acquisition of procedural skills for minimally invasive surgery (2). In the late 1990, the first national pedagogical problem arose with the growing impact of laparoscopic surgery. A committee was in charge to develop educational material to «teach a standard set of cognitive and psychomotor skills to practitioners of laparoscopic surgery» to address this problem. Consequently, the (Fundamentals of Laparoscopic Surgery (FLS) curriculum was developed as a high stakes exam (HST) for laparoscopic surgery (3), however there was no official curriculum for the HST. Surgeons need a specific evaluation that allows them to assess their ability and technical skills. Subsequently, multiple consensus conferences were held by the Fundamentals of Robotic Surgery Group (FRS), the Department of Defense and Veterans Administration and 14 Surgical Specialty Societies, in order to develop a comprehensive model for robotic surgery. The consensus conferences consisted of over 80 subjects matter experts (SME), including surgeons, psychologists, psychometricians, engineers, simulation experts and medical educators (4). Two principles guided the committees development process of the FRS: first to ensure a perfect understanding of the basics of robotic surgery including the use of the da Vinci robot and second to achieve a psychomotor skills program focused on the tasks specific to robotic surgery. To accomplish these objectives, at the end of 2011, the FRS committee convened an international body of leaders in robotic surgery to define the skills necessary to begin the process of creating a Previous Version Date: Page 5 of 52

6 certifiable curriculum (Outcomes Measures and Metrics consensus conference). This curriculum included cognitive, psychomotor, and team training skills. This resulted in a prioritized matrix of 25 specific robotic surgery tasks that was the core material to develop and design the FRS Curriculum (5). As the purpose was to develop a basic fundamental skills program, particular references to an anatomic location or to a specific procedure have been avoided. The overall goal of the FRS program is to teach a standard set of cognitive, psychomotor and team training skills in robotic surgery (3). The clinical objective is to provide surgeons with the necessary fundamental knowledge and training, thus ensuring a standard of care to patients undergoing a robotic procedure. An online curriculum and a dry lab for psychomotor skills training were developed with two types of assessment tools: knowledge assessment using an online didactic curriculum (reviewing the robotic platform and team training skills) with quiz and psychomotor skills assessment using a novel device with technical skills examination. During the process, multiple Delphi reviews were done to control the adequacy of the learning module and to ensure the consistency between the FRS curriculum and the initial 25 most important items related to robotic surgery. The psychomotor skills test is designed to be a physical model however this presents several disadvantages such as increase cost of the training (i.e. required time for use of da-vinci robot, and training instruments). Additionally, many training centers do not have a da Vinci robot dedicated to training which would be needed in order to train with the FRS psychomotor skills device. A possible solution to circumvent the physical model would be to develop an equivalent psychomotor skills test using a virtual reality simulation platform. Simulators have demonstrated validity in robotic surgery training (6,7,8), the two commercial leaders of robotic simulation have designed virtual skills exercises for training with the da Vinci Skills Simulator - DVSS (Intuitive Surgical Inc., Sunnyvale, CA) and on the dv-trainer - DVT (Mimic Technologies, Inc., Seattle, WA). The FRS Online Curriculum has been created in the form of videos and text, which provide basic instructions on the use of the da Vinci robot from the first preoperative tasks, through the end of post-operative tasks. It summarizes the necessary steps for a safe, successful robotic operation, the identification and minimization of critical errors that may arise during the procedure. It also includes a module on team training and communication skills. All tasks are to be completed to a benchmark level of proficiency, which will be based on the performance of expert robotic surgeons during the larger multi-site validation trial. The didactic section of the FRS Curriculum includes proficiency-based multiple choice tests that serve to evaluate knowledge of the most important aspects of using the robotic surgical device such as general robotic system knowledge, use of the instruments and troubleshooting (9). The FRS Psychomotor Skills device evaluates the most important basic tasks in robotic surgery. The FRS committee developed a progressive scale of 25 items (Table1) from which seven FRS Psychomotor Skills were created (Fig. 1). Previous Version Date: Page 6 of 52

7 Fig 1: The FRS Dome from the virtual to the physical. The seven different tasks of the Psychomotor Skills Curriculum include: 1) Docking and Instrument Insertion, 2) Ring Tower Transfer (Endowrist manipulation), 3) Knot Tying, 4) Railroad Track (suturing exercise), 5) 4 th Arm Cutting (Multi-arm control), 6) Puzzle Piece Dissection (dissection and a-traumatic handling), and 7) Vessel Energy Dissection (dissection, energy source control). A 3-D dome was designed such that it would assess these skills in a dry lab format. First a virtual dome model was created via graphic animation to depict the seven tasks and the physical aspects of the dome such as the differents layers (skin, fat and vessels) (Fig. 2). Several physical prototypes were built and tested and underwent multiple revisions in order to achieve a realistic and successful design that is adequate for testing and evaluation. Table 1: The 25 fundamental items in robotic surgery Pre-Operative Intra-Operative Post-Operative System Settings Energy Sources Transition to Bedside Asst Ergonomic Positioning Docking Camera Control Clutching Undocking Robotic Trocars Instrument Exchange OR Set-up Situation Awareness Foreign Body Management Multi-arm Control Closed Loop Comms Eye-hand Instrument Coord Previous Version Date: Page 7 of 52

8 Respond to System Errors Wrist Articulation A-traumatic Tissue Handling Dissection Fine & Blunt Cutting Needle Driving Suture Handling Knot Tying Safety of Operative Field a b) c) Fig 2: Components of the FRS dome: basic structure (a), fat with vessel (b), skin (c) Previous Version Date: Page 8 of 52

9 Once the final prototype was created, a computer-based virtual reality simulation of the final FRS psychomotor skills dome was developed for the DVSS and DVT simulators. The multi-site validation trial, which is to follow to this pilot study, will collect data on the degree to which these two virtual reality devices provide training outcomes are equivalent to those with the use of the real robot on the physical dome (Fig 3). As the larger validation trial will include this comparative effectiveness arm of validation, this smaller pilot study also includes these arms in order to identify any issues that might be encountered in the final trial. a) b) Fig 3. Virtual Reality simulation of the FRS Dome: Virtual DVSS Dome (a) and Virtual DVT Dome (b) Study Objectives The aim of this study is to conduct a pilot test of the learning materials and assessment tools of the FRS Curriculum to establish preliminary validation of the curriculum and its primary objective and to identify any necessary modifications to the study prior to allocating large funds and resources for the official multi-institutional validation trial. Primary Objective/Aim/Goal/Hypothesis The primary objective of this study is to study the feasibility, practicability and usability of the FRS Online (shown in Figure 1) and Psychomotor Skills Curriculum (device shown in Figure 1), and to provide baseline data that will help guide the official validation trial of the FRS. In addition, preliminary validation information will be gained with SMEs reviewing the curriculum for face & content validity and comparing performances of novice and expert groups for construct & concurrent validities (10). For Psychomotor Skills we will perform the preliminary comparative effectiveness of training and assessment on the real FRS Dome versus the two virtual dome versions of the exercises of the DVSS and DVT simulators. Secondary Objective/Aim/Goal/Hypothesis The secondary objective of this study is to conduct a preliminary trial for setting the benchmark proficiency level that could be expected from expert performances for the online curriculum and the psychomotor skills. This will give insight as to whether training and assessment on the virtual DVSS and DVT domes are equivalent to performance on the real FRS dome. The implication is that if the virtual systems demostarate that they are equal to real systems it is possible to Previous Version Date: Page 9 of 52

10 significantly reduce cost, time, resources and personnel in training and assessment through implementation of virtual reality simulation surgical training. Face and content validities are more effective in evaluating the ability of the FRS Curriculum to train (not to assess) a surgeon, while construct and concurrent validity are geared to evaluate the FRS Curriculum effectiveness to assess the trainees (10). Face validity is assessed by experts and is used to determine the realism of the FRS Curriculum, or whether the dome represents what it is supposed to represent. Content validity is the assessment of the appropriateness of the FRS Curriculum as a teaching modality and involves formal evaluation by experts knowledgeable about robotic surgery. This determines whether the FRS Curriculum can realistically teach what it is supposed to teach. Construct validity is the ability of the FRS Curriculum to differentiate between the performances of experienced and non-experienced or less-experienced users. Concurrent validity is achieved when a test correlates well with a previously validated measure that is defined as the gold standard. As a gold standard does not exist to compare to the FRS curriculum, performing the 7 tasks on a tissue model will represent a real model of comparison for the FRS Psychomotor Skills. Reliability and predictive validity will not be included in the first step validation. Study Design Research Design This is a randomized prospective study designed to assess the feasibility, practicability and usability of the FRS Curriculum, both, online and of psychomotor skills including experts and novices. In addition, data collected will provide information on the magnitude of the parameters of many of the aspects of the full official Validation Trial, such as the likely level of the benchmark for proficiency, the standard deviation of performance, the number of trials for experts to achieve benchmark proficiency. All subjects in the experiments will be categorized based on their number of years of surgical experience in robotic surgery, the total number of robotic procedures performed, and their previously reported weekly case load in robotics. These will be referred to as the Expert Group (EG) and Novice Groups (NG). Each of these two groups will be divided into 3 sub-groups of the Psychomotor Skills program: 1) Real FRS dome; 2) Virtual DVSS Dome; 3) Virtual DVT Dome. Inclusion criteria: - Expert Group (EG): Robotic surgeons with more than 100 procedures performed. - Novice Group (NG): Fellows, residents, attending surgeons without experience in robotic surgery or with 10 or less robotic procedures in which they performed more than 50% of the procedure. Previous Version Date: Page 10 of 52

11 Six experts and twelve novices will be included in this experiment. Novices and Experts will be assigned in a random order to each subgroup. For novices and experts we will employ a restricted randomization using the random allocation rule. This will insure that each treatment has the same number of subjects experiencing each version of the online curriculum. This is important for the objectives of this pilot study with a very small number of novice subjects. Two experts and four novices will be included in each Psychomotor Skills sub-group. Subjects identity will be coded using a unique subject ID number to ensure data confidentially. Each participant, under the ID Number, will complete the demographic data questionnaire provided in Appendix 1. All data will be de-identified and the data entered into a spreadsheet for analysis. Exclusion criteria: - EG: Robotic surgeons who are not able to complete all the procedures and experienced laparoscopic surgeons with no robotic surgery experience may be excluded. - NG: Residents, fellow, attending surgeon with more than 5 robotic cases done independently, and those who are not able to complete all the procedure. Curriculum to validate: This pilot study will direct the subjects through the use of two different sets of online curriculum designed to familiarize a future student with the necessary information and knowledge for becoming a credentialed robotic surgeon. This version of the curriculum which has been created by Intuitive Surgical Inc. has been the standard for educating robotic surgeons for seven years. This curriculum is located on the web at: The pilot study will compare a new online curriculum developed by the Fundamentals of Robotic Surgery project (a collaboration of 14 societies and 50 scientists) to the established curriculum. This new curriculum can be found on the web at: Both online systems require a personal account to login to the curriculum. The accounts can be requested from the owners of the material on their web sites. 1. The 4 modules of the Online Curriculum: This online curriculum, composed of 4 modules, covers the different steps of a robotic procedure. This curriculum is hosted in an online learning management system, which will track the performance of all subjects who log-in with their assigned subject ID. Previous Version Date: Page 11 of 52

12 Module I o Robotic system advantages o Components of Robotic system o System functionality Module II o Pre-operative phase o Intra-operative phase o Post-operative phase Module III o Explanation of the Psychomotor Skill Curriculum develop with the dome s exercises Module IV o Team training & communication skills Face and content validities: this part of the study will be achieved during the Delphi process, during which experts perform the online curriculum and record personal suggestions to improve the curriculum. These suggestions will be included in the final report for DoD and which is also is forwarded to the FRS Executive Committee and the Principle Investigators of the FRS. Construct validity will be achieved comparing EG and NG performances on the quiz questionnaires. Fig 4: Diagram of the construct validity study design of the FRS online curriculum. The expert performance will provide a basis of the proficiency level (passing score). The goal for novices is to reach the same level of expertise. Since this is a pilot study, the proficiency levels achieved by the small sample of experts in the sample are used only to guide the conduct of the pilot and to complete all of the necessary activities. The real proficiency level will be established in the larger official multi-institutional study. Concurrent validity will be achieved comparing the FRS online curriculum to the currently most used curriculum, which are the materials created by the robotic device manufacturer (Intuitive Surgical Inc.) and which have been used to teach all robotic surgeons up to this point in time. This curriculum has not been academically validated, but it has been the basis for training all surgeons since the robot was first approved for surgery. Previous Version Date: Page 12 of 52

13 Design study: The novice group will be randomly split into two sub-groups of novices. Novices Group A: will perform the FRS online curriculum and take the FRS online quiz. Novices Group B: will perform the Intuitive Surgical online curriculum and also take the FRS online quiz. We will compare the performance of both groups on the FRS quiz and then both groups will undergo an assessment test in a tissue model procedure to compare the knowledge achieved by each group. This test represents their baseline performance before beginning the psychomotor skills. For the team training skills, they will also perform an initial da Vinci robot docking procedure to assess the knowledge transfer of the online learning content on psychomotor and team-training/communication skills to a practical process using an operative checklist (Appendix 2). Fig 5: Diagram of the concurrent validity study design of the FRS online curriculum 2. The 7 tasks of the Psychomotor Skill Curriculum: The second part of the curriculum is the use of the psychomotor skills device, usually referred to as the FRS Dome. This device supports the following exercises. The Psychomotor Skills Exercises and Metrics are explained in Appendix 6. Task 1: Docking and Instrument Insertion (Fig.6) Subjects will realize the docking task introducing the trocars for the camera and instruments into in the white-box trainer. The dome will be in the closed box. Previous Version Date: Page 13 of 52

14 Fig 6: Task 1- Docking Task 2: Ring Tower Transfer (Fig.7) Four towers with a S-wire are positioned on the dome, two towers on the center cap and two on the lower part (one on the left side and one on the right side). There is an 8 mm ring on the base of each tower located at the upper part of the dome. Using both hands, subjects will transfer the ring from the S-tower on the cap to the S-tower on the skin surface, twice. Fig 7: Task 2 - Ring Tower transfer. Task 3: Knot Tying (Fig.8) Subjects will tie a surgeon s knot to approximate the two eyelets of the I towers such that they touch each other and back up the knot with a square knot (two throws). Previous Version Date: Page 14 of 52

15 Fig 8: Task 3 - Knot Tying Task 4: Railroad Task (Fig.9) Subjects will perform horizontal mattress suture through the target points to approximate the tissue. Fig 9: Task 4 Railroad Task Task 5: 4th Arm Cutting (Fig.10) Subjects will pickup and stretch the elastic band with the first and third arms and use second arm instrument to cut at the marks. Place the cut sections in the bowl on top. Fig 10: Task 5-4th Arm Cutting 1 st cut 3 rd cut 2 nd cut Task 6: Puzzle Piece Dissection (Fig.11) Subjects will cut out and dissect the shape marked on the skin. Previous Version Date: Page 15 of 52

16 Fig 11: Task 6 Puzzle piece dissection Task 7: Vessel Dissection (Fig.12) Subjects will dissect through the fat layer to expose the vessel, cauterize and cut the vessel. Fig 12: Task 7 Vessel Dissection Face & content validity of the Psychomotor Skills will be evaluated by subjective opinion of experts. Experts, under their ID Number, will evaluate face and content validities of the FRS Psychomotor Skills Curriculum with a specific questionnaire: face and content validity questionnaire (Appendix 3). All de-identified data will be entered into a spreadsheet for analysis. Experts will also assess the effectiveness of the FRS Tasks to ensure the adequacy between the FRS Psychomotor Skills and the initial 25 most important items related to robotic surgery answering to the questionnaire provided Appendix 4. Construct validity will be evaluated comparing the ability of expert and novices in each sub-group (Fig.13). FRS real DOME: For the experts & novices in the FRS real Dome sub-groups, an outside video of the task 1 (docking) and the da Vinci robotic view will be recorded for each subject performing the FRS Psychomotor Skills. The videos will be de-identified with the ID Number, and evaluated using the validated GEARS protocol (Global Evaluative Assessment of Robotic Previous Version Date: Page 16 of 52

17 Skills) (provided in Appendix 5) (11). [Note that the experienced surgeon evaluators are not the same surgeons who are serving as the expert group for this study.] This will provide each subject an individual task scoring and a global score. We will ask the EG to perform the Psychomotor Skills until they reach a proficiency level (2 consecutive trials without improvement of the score). This level will be the goal to reach for the sub-group of novices. FRS virtual DOMES: For the expert & novice in the virtual DVSS Dome sub-group, the simulator will provide an automatic scoring of the subject in a number of different performance metrics. The specific metrics of interest are described later in the analysis section of this protocol. Experts will perform the 7 exercises until they reach proficiency (2 consecutive trials without improvement of the score). This level will be the goal for this novice's sub-group. For the expert & novice in the virtual DVT Dome sub-group, the simulator will provide an automatic scoring similar to that in the DVSS simulator. The experts will perform the 7 exercises until they reach proficiency (2 consecutive trials without improvement of the score). This level will be the goal to reach for this novice s sub-group. We will compare the results of the first trial between each sub-group of experts with its matched sub-group of novices to evaluate preliminary construct validity. Concurrent validity for Psychomotor Skills curriculum will be compared to a similar evaluation on a tissue model. Each novice will perform 7 tasks on a tissue model (t b ) as their baseline performance level prior to the Psychomotor Skills curriculum. After completion of the Psychomotor Skills training to proficiency, they will perform the same 7 tasks on the same tissue model (t f ). Experts will only perform the final trial (t f ). Each procedure will be recorded, deidentified, assigned a unique identifier, and evaluated using the validated GEARS protocol (11). This will provide to each subject individually real tissue task scoring and a global score (Fig.14). Previous Version Date: Page 17 of 52

18 Comparison on the first trial between each subgroup = Construct Validity ToT Fig 13: Diagram of the FRS Psychomotor Skills validation for expert and novice groups (ToT: Transfer of Training). Previous Version Date: Page 18 of 52

19 Real tissue model: We designed the 7 Tasks on a tissue model: a turkey thigh. We will place it in a closed white-box trainer designed for robotic education. A short and standardized explanation of the different tasks to perform will be given to each participant before they start the process (provided in Appendix 6). Task 1: Docking and Instrument Insertion (Fig.14) Subjects will realize the docking task introducing the trocars for the camera and instruments into in the white-box trainer. The turkey thigh will be in the box. Task 2: Ring Tower Transfer This task assesses the Endowrist manipulation, eye-hand coordination, clutching and camera control. A real procedure that looks like the Ring Tower Transfer is not available on a real tissue model. We designed a task that emphasizes Camera control. Endowrist manipulation is covered in the Puzzle piece dissection and in the suture exercise. The subject will clearly focus the camera on 3 small targets drawn on the inner face of the thigh. To adjust the target, the subject will use the clutch (Fig. 14). Task 3: Knot Tying The subject will perform a knot with the thread previously positioned on the inner face of the thigh (Fig.15). Task 4: Railroad Task The subject will perform a running suture on a 4 cm skin incision in the outer face of the thigh (Fig.14). Task 5: 4th Arm Cutting The subject will finish-cut a pre-cut skin flap on the outer face of the thigh, will expose it using the 4 th arm, and will cut it on designated marks (Fig.14). Task 6: Puzzle Piece Dissection The subject will dissect a puzzle piece tattoo on the inner thigh skin without muscle injury (Fig.15). Task 7: Vessel Dissection The subject will dissect femoral vessels on the inner face of the thigh and coagulate them (Fig.15). A novice assistant will be present to help the subject in the docking task, but he will not provide any directive assistance, only what is specifically requested by the subject. Two videos will be recorded - one, an external view of the Operating Room at the beginning of the procedure (to evaluate docking) and one of the da-vinci robot view for the entire procedure (to evaluate skills in performing the tasks). Previous Version Date: Page 19 of 52

20 Fig 14: Outer face of the thigh with Task 2, 4, 5 and 6. Fig 15: Inner face of the thigh with Task 3 and 7. Study Agent, Device, and/or Intervention Description Not applicable Study Site(s)/Location(s) and Number of Subjects Florida Hospital site locations: Nicholson Center, Celebration, FL Estimated number of subjects at Florida Hospital sites: 18 Multi-Site Research Logistics/Communication Plan Not applicable. Note, this pilot study is evaluating the process for a future multi-site study. But the pilot will not be conducted as multi-site. Community-Based Participatory Research Previous Version Date: Page 20 of 52

21 Not applicable. Subject Selection Vulnerable Populations Employees are considered a vulnerable population. To reduce the prospect of coercion, the research team will discuss the study and consent all potential subjects. Subjects will be ensured that their participation or non-participation will have no effect on his/her relationship with Florida Hospital, Administration, or be reflected in his/her performance evaluation. Inclusion Criteria 1. To be currently a physician. Physicians may be surgical residents, fellows or attending surgeons. Verification of physician or medical student status may be done by checking University ID (for medical students), hospital or other related practice ID (for physicians). If the individual does not have this ID on them, a verbal affirmation of their current status as medical student or physician will suffice. 2. Is willing and able to comply with study procedures design. 3. Subject will be able to provide informed consent Exclusion Criteria - EG: Robotic surgeons who are not able to complete all the procedures and experienced laparoscopic surgeons with no robotic surgery experience may be excluded. - NG: Residents, fellow, attending surgeon with more than 5 robotic cases done independently, and those who are not able to complete all the procedure.. Study Procedures Site Initiation/Study-Specific Training Not applicable, Subject Recruitment and Screening Study introduction/advertisement will be done using IRB approved flyers. The investigators/designated research associate may also approach potential candidates with approved materials. A copy of the informed consent will be provided to each potential subject so that they can take time to consider participation. Once the decision has been made that the individual meets the inclusion criteria, s/he will be invited to participate in the study and will be scheduled to come to the site to initiate their participation. Consent Procedures If the subject (resident, fellow, attending surgeon or expert surgeon in robotic surgery) would like to participate in the study, he will be given a copy of the informed consent to review. The subject will be informed that their participation in the study is voluntary. Sufficient time for review of the informed consent paperwork will be given prior to answering questions about the study. The subject and the person obtaining informed consent will then Previous Version Date: Page 21 of 52

22 sign the informed consent form. Subject will be provided with a copy of the signed and dated consent. Non-English Speaking Subjects Not applicable Waiver of Written Documentation of Consent or Waiver of Consent Not applicable Documentation of Informed Consent Process Documentation of the informed consent process is required to establish that the subject was accurately and adequately informed and that no study-related procedures were initiated prior to obtaining informed consent. A research team member will note in the source documentation the consent process, date consent was obtained and that consent was obtained prior to initiating any research procedures. Randomization This is a pilot study designed to evaluate the procedures that will be used in a larger multistudy to be performed in the future. As such, the subject sample size is small and it is important that all arms of the study be tested equally. Therefore, subjects will be assigned to a specific treatment using restricted randomization using the random allocation rule. This will insure that each treatment arm has the same number of subjects. The statistical software, SPSS will be used for the analysis of the data. For the Online curriculum - Experts will be all assigned to the FRS curriculum. - Novices will be assigned to the FRS curriculum (Novice A) and Intuitive Surgical Inc. curriculum (Novices B). For the Psychomotor Skills: - Experts will be randomly assigned to one of the three groups: real FRS Dome; virtual DVSS Dome; virtual DVT Dome - Novice A and Novice B will be randomly assigned to one of the three groups: real FRS Dome; virtual DVSS Dome; virtual DVT Dome; This information will only be released as the participants are enrolled. A master list of the subjects names and intervention group will be kept in a secure location by the Celebration Research department. Subject questionnaires and performance data will be coded to de-identify the subjects, enter data into a spreadsheet and lead the subjects through the exercises. The investigators will not have access to this code until the end of the study. Study Visits Visit 1: Inclusion and exclusion criteria will be reviewed. If the subject meets criteria, s/he will be invited to participate in the research study. The subject will be given the informed consent and it will be reviewed and all questions will be answered. After the Previous Version Date: Page 22 of 52

23 informed consent is signed, the subject will be randomized to one of two novice groups or to the expert group according to their experience in robotic surgery. Instruction for visits to our centers will be given and contact information will be given to them to schedule their appointment before the visits. The study design will be explained to them. Study Duration The study will start two weeks post-irb approval and DoD approval. We anticipate completing enrollment within a 3 month time period. Rolling admission for 18 subjects should take approximately 12 months to complete the data collection and analysis portions of the study. The online curriculum may be completed in 2 weeks to ensure the understanding for novices and the quality of the reviewing for experts. During the next four weeks subjects will be allowed to perform the psychomotor skills on the domes (real FRS; virtual DVSS; virtual DVT) and on the real tissue model (one for expert twice for novice). Materials of Human Origin: Collection, Preparation, Handling and Shipping Not Applicable Study Outcome Measures (Endpoints) Through this study, we expect to establish feasibility, practicability and usability as well as Face, Content, Construct and Concurrent validity of the FRS curriculum, to be sure to develop a valid, comprehensive program of education and assessment of the basic fundamentals of robotic surgery. We also want to perform a preliminary validation of both virtual DVSS and DVT Dome simulation exercises comparing the performances of experts on the real tissue model trial (t f ) after training on the real FRS Dome and on simulators. From novice performances, we expect to study Transfer of Training (ToT) and Training Effectiveness Ration (TER) between the two trials on the real tissue model (t b ) and (t f ) after training on the Dome (Real FRS; virtual DVSS; virtual DVT). Data Management and Quality Plan Data De-identification Four types of data will be provided: (1) Data collected via questionnaire: Demographic data & Validities questionnaires (2) Data collected by the online curriculum system, which includes performance on quizzes and the number of times a subject goes through the materials. (3) Subjects performance data that will be collected via recorded videos of Psychomotor Skills curriculum and real tissue model performances for each subject. (4) The resulting "skin" and "fat" from each subject's psychomotor skills exercises device, which will be retained. Previous Version Date: Page 23 of 52

24 Each subject enrolled in the study will receive at the enrollment a unique identification number. It will represent the order at which they are enrolled in this study and the group (expert & novice) and sub-groups the subject will be randomly assigned. This number will be used throughout the study to de-identify all data. The key to this coding will be maintained in a limited access, separate, password protected data file. The data collected from the questionnaire will be transcribed into electronic format (excel spreadsheet) and referenced by the unique subject identification number. These spreadsheets will be periodically sent to the investigator. The videos will be also de-identified. The performance data analyzed with the GEARS protocol will be exported and include in a spreadsheet after blinded evaluation. This data will be automatically sent to the PI after each session. Both of these data will be gathered in a master spreadsheet or database that will be updated regularly by the investigator until the end of the experiment. It will be the responsibility of the all investigators to maintain the confidentiality of the data. The records will be retained in a limited access, password protected computer for a minimum of 7 years post completion of the study. Participants identity will be coded using a unique subject ID number. Numbers will be assigned consecutively upon enrollment. The master list for the key to the unique ID code will be maintained in a limited access, separate, shared file that will be made available to study team members only. Demographic data will be categorized to the extent possible to further protect confidentiality of subjects. Data Confidentiality, Storage, and Retention All study related paper documentation including the Investigator Binder, completed informed consents, and subject records will be maintained in separate limited access files in locked rooms. Electronic data will be stored in a limited access, password protected database overseen by the Research Department. Access will be further restricted to only those study team members who have been designated by the PI to have data entry capability. Data Quality The PI is responsible for reporting any reasons outside the planned study design such as noncompliance with the protocol or any delay in the initiation of the study due to administrative reasons. The PI and designated CH- Research Institute staff will conduct ongoing internal monitoring of the resultant data to assure data integrity. This includes - 100% review of eligibility and consent related matters. - Additional quality control procedures for this research study include source data verification by randomly selecting 10% of subject records with comparison between the paper survey forms and the electronic database Previous Version Date: Page 24 of 52

25 record. If errors are common, all data will be completely checked prior to data analysis. Data Sharing (outside of Florida Hospital) Not Applicable Sample Size Determination The sample size for this experiment is driven by the nature of a pilot study, the number of treatments in the study, and the number of subjects that are available with the necessary background. Based on previous and currently ongoing experiments, we estimate that the subjects available for this study will be 18. Statistical Analysis Plan Since this is a pilot study, the scientific results and statistical analysis will be more limited than in a full study. Through the use of this pilot with a small number of subjects, we hope to accomplish each of the following: Primary Objective Analysis 1. Evaluate the usability, feasibility and reliability of the processes, which will be used in a larger, comprehensive multi-institutional study. Examine each step for processes in the protocol, to insure there are no incorrectly or poorly designed portions. Results will be used to inform and modify the protocol for the larger study. No statistical analysis is required for this objective. 2. Arrive at a rough estimate of the time that will be required for a subject to perform all of the activities that are assigned in the protocol. Use this information in the scheduling and planning for the larger study. For this, basic statistical analysis of mean times and standard deviations will be calculated. 3. Arrive at rough estimates for the number of repetitions of each exercise that will be required for a novice to reach proficiency in each exercise. The number of repetitions will also assist with planning the duration of the larger exercise and perhaps the number of return engagements that should be used for subjects (as opposed to expecting them to perform all repetitions at a single visit). For this, basic statistical analysis of mean times and standard deviations will be calculated. Secondary Objective Analysis 4. Arrive at a rough estimate for the proficiency thresholds, which will be set by expert surgeons in the various exercises. For this, basic statistical analysis of mean times and standard deviations will be calculated. 5. Evaluate the effectiveness of the tissue model selected as a surrogate to demonstrate proficiency with the actual robotic device. No statistical analysis is required for this Previous Version Date: Page 25 of 52

26 objective. Given the nature of this pilot study, the basic statistical analysis of the results described above will be all that is performed. This information will be used to design a more effective protocol for a larger multi-site protocol and validation trial. Potential Risks and Benefits Potential Benefits There is an associated financial gain for the participants in the form of a subject stipend which is paid through the different phases of the experiment. For details see Subject Stipends or Payments Potential Risks There are no potential risks. Potential Data Risks Every effort will be made to protect the confidentiality of the subject and resultant data. This includes using a unique subject identification code and limited access databases. Provisions to Protect the Privacy Interest of Subjects Subjects will be assigned unique identifiers for study-related records. All precautions will be taken to make sure that only authorized individuals will access subject research records. The collection of sensitive information about subjects will be limited to minimum necessary to achieve the aims of the research, so that no unneeded sensitive information will be collected. Early Withdrawal of Subjects Investigator Withdrawal of Subjects Subjects may withdraw consent from the study at any time without consequence. Those who withdraw or are withdrawn will be replaced. Subjects that did not complete the full course of the FRS Curriculum will be removed from the study. Subjects that do not complete at least 100% of the investigational procedures will be determined to be dropouts and separated out for analysis purposes. These subjects will be replaced. Once enrollment has ended, study subjects will not be replaced. Subject Request for Withdrawal from Study Subjects may withdraw consent from the study at any time without consequence. New subjects will replace subjects who drop out before completing the process when such new subjects are available. These new subjects will start at the beginning of the sequence. All subjects that do not achieve the complete protocol study will be excluded and data not exploited. However, to allow for replacement of dropouts and withdrawals, we will leave enrollment open for a 12 months. Data Collection and Follow-up for Withdrawn Subjects Previous Version Date: Page 26 of 52

27 Subjects who request withdrawal or who are withdrawn by the PI from the study will have their data maintained in the research database up to the point of withdrawal. This data will not be included in subsequent analysis. Adverse Event Reporting: Adverse Events Not applicable. Recording of Adverse Events Not applicable. Notification of Adverse Events Not applicable. Safety Monitoring Plan Safety Monitoring The PI is responsible for reporting any reasons outside the planned study design such as noncompliance with the protocol or any delay in the initiation of the study due to administrative reasons. Data and Safety Monitoring Board (DSMB) or Equivalent Not applicable Ethical Considerations Employees/ Students are considered a vulnerable population. To reduce the prospect of coercion, we will discuss the study and consent all potential subjects. Subjects will be ensured that their participation or non-participation will have no effect on his/her relationship with Florida Hospital, management, or be reflected in his/her performance evaluation. Sharing of Results with Subjects Individual results will be shared with the subject, on the online curriculum with an automatic scoring provide at the end of the quiz question and on the simulator dv-trainer, giving them explanation on their overall score and criterion scores. Funding Source This study is being funded by a grant from the United States Army Medical Research and Materiel Command (USAMRMC) and its laboratories identified herein through the U.S. Army Medical Research Acquisition Activity (USAMRAA). Conflict of Interest Neither the principal investigator nor any other parties involved in conducting this research study have patent ownership, receive royalties, or will obtain any financial gain. Subject Stipends or Payments Subjects will receive a financial compensation for taking part in this study. An analysis of the Previous Version Date: Page 27 of 52

28 level of effort required by the subjects to complete the study determined that each subject would need to spend approximately 8 hours to complete the entire experiment. This is approximately 4 hours working through the didactic educational curriculum and 4 hours of direct usage of the da Vinci robot for the psychomotor skills device. Through discussions with both the ORA and IRB leadership we determined that $500 would be an appropriate total stipend for each subject completing the study. To eliminate undue coercion for subjects to remain in the study, this amount is divided into four pieces which accrue to the total $500. Specifically, we will compensate each subject as follows: 1) Didactic Curriculum, Modules 1 & 2. Subjects will receive $125 for completing Modules 1 & 2 of the didactic curriculum. 2) Didactic Curriculum Modules 3 & 4. Subjects will receive $125 for completing Modules 3 & 4 of the didactic curriculum. 3) Psychomotor Skills Engagement. Subjects will receive $125 for arriving at their scheduled appointment for the skills exercises (robotic or simulation as assigned) and performing each exercise one time. 4) Psychomotor Skills to Proficiency. Subjects will receive $125 for arriving at their scheduled appointment for the skills exercises (robotic or simulation as assigned) and performing each exercise until receiving two passing scores on each. Each subject s progress through these four milestones will be tracked. Each subject will receive one payment for all accomplished milestones upon completion of the study or withdrawl from the experiment. Publication Plan The results and progress of all of this research work will be prepared for publication in medical journals, engineering or computer science journals and presented at relevant conferences. All data will be reported in aggregate to protect the identity of all participants. Previous Version Date: Page 28 of 52