Orthopaedic In-Training Examination FAQs

Transcription

1 Orthopaedic In-Training Examination FAQs AAOS Evaluation Committee AAOS Evaluation Programs Howard Mevis, Director, Laura Hruska, Manager, Examinations Marcie Lampert, Senior Coordinator, Examinations Anna M. Scheer, Coordinator, Examinations Denise Plasky, Assistant, Examinations Irene Bogdal, Administrative Assistant, Examinations Department of Electronic Media, Evaluation Programs & Course Operations & Practice Management Group Updated 2010/10

2 Frequently Asked Questions (FAQs) Software Features Q: Does the tutorial need to be used to install on each computer or will each disc be installed that day? A: The tutorial is part of the software and does not need to be installed separately. Residents will have access to the tutorial during the examination. Q: There are videos on the test; do we need sound for the test? Head phones? A: There will not be sound on this examination. The sound was included on the 2009 demo disc to showcase the possibility. Q: I read something about a built-in proctor? A: The software has built-in proctoring features. The software will automatically shut down the examination (and timer) if you access any of the tool bars either on top or bottom of the window. If you go to close a pop-up window, check , try to lock your computer, or try to access any other applications, the software will shut down the examination and will require a proctor code to re-enter. Q: What, if any, other programs can be open? A: The way the software works is that it must be the one and only program running, The Main Event. If you go to minimize the window, it will think you are trying to view other materials and the software will automatically shut down. Q: Do the residents have to log in online? Can we use laptops using the WiFi for UF building? A: No, the examination is disc-based. The 2010 OITE is not an online examination. The disc must be in the actual computer for the resident to see the question and image. Q: Will we be able to use an Apple computer? A: Yes, the software is designed to run on both PC and MacIntosh (Apple) whose operating system is 10.4 or better. Q: While reading through Chapter Four, It states that if you close a pop up that appears on your computer, the test will automatically close down. My laptop always has a ton of pop ups - how am I supposed to read the screen with pop ups that I'm not allowed to close in my way? A: It is true that if you go to close a pop up in examination mode the exam will shut down. You must work off-line to avoid getting any pop ups.

3 Installing the Software Q: Can I use the same disc that I initially install it on 16 computers to reinstall after I remove the program following the first set of residents who took it? A: Yes. If you use one disc (for example the extra disc sent to every program) to install on the 16 computers for the first set of residents who take the examination, each who has a disc in the specific machine on the day of the examination. Then you would need to remove the software and the program folders from the 16 computers. You could use that extra disc to reload each of the 16 computers. Again, during the actual examination each resident must have a disc in the computer disc drive. The reason why you can use one disc to load all the computers is because you are only installing the software to run the examination. The actual questions and images are on the disc and that is why a disc must be in every computer during the test. Q: Will our program receive any extra discs just in case of a snafu? A: Yes, every program will receive two extra discs in your shipment. Q: How long does it take to load the exam? How long does it take to register the disc? A: It will take about 2 to 3 minutes to load the software on the computer. The disc themselves are not registered. The resident will register themselves on the day when they take their examination which should take approximately 3 minutes to complete the necessary information. Q. If we install the DVD-ROM onto the residents' computers beforehand, will they have access to the test before the November 14th test date? A. No, as suggested in this Chapter it is a good idea to pre-install the software so you know the program is working. Take the disc out and the resident will not have access to the examination. Q: Can our IT staff call you directly? A: Yes, it would be a good idea to have IT call us early if they have any questions. Q: On page 4 of Chapter 1, it says "Your OITE program uses Flash version 6.0" Then in Chapter 6, page 4, it says "Adobe Acrobat Reader 9. Which is correct? A: Both. The program Flash version 6.0 is a very early version of the Flash program which is needed to run the Program Tutorial. Most computers will have this version of Flash software. Adobe Acrobat Reader 9 is a different program that is also free and is used for the OITE Help Guide. When the resident has completed the examination and submitted their answers, they can re-enter the OITE and link to a print version of the OITE examination questions. The OITE Help Guide which is available during the examination from the [Help] menu is a PDF document which is read with program Adobe Reader 9. Q. Can the IT department use the complimentary disc for a confidential test run? A: Yes; and we encourage your IT department to call any of the phone numbers listed on the Technical Support Card with any questions during this time prior to November 13, Frequently Asked Questions /OITE DVD; 2010/10 Page 2 of 11

4 Test Day Q: Will everyone get a CD? A: Yes. Each resident will receive a CD-ROM. The program will also receive two complimentary copies for faculty use in post-examination teaching activities. Q. Should we use a backup? A: Yes, there is a complimentary answer sheet in the OITE booklet inside each CD case. The residents can use a pen or marker to check their responses. This back-up with their answers would be available to them immediately while the review mode code is not released and if there was any computer malfunction. Q. Can residents use pen and paper to take notes? A: This is the decision of your program director to allow residents to bring scratch paper. Q: If a computer shuts down, do I give the resident the proctor code to unlock? A: The proctor should be the individual to enter the proctor code. Q. Whose address should go in the registration page? A: No address will be required for the 2010 OITE. It is imperative that the residents correctly enter their program ID and their AAOS ID # as indicated on the labels provided so that they can receive credit for their examinations. Frequently Asked Questions /OITE DVD; 2010/10 Page 3 of 11

5 Q: Before a resident can confirm their answers, they must answer all questions? What if they run out of time? A: Yes, all questions must be answered before the resident can complete their exam. If they try to go to the confirmation screen and there are unanswered questions then they must go back and complete those questions. However, if they run out of time, just like the Board Qualifying Examination, they will not be able to complete their responses. Q: For lunch breaks or to answer a page, what does the resident need to do? Can they turn off the monitor so no one sees their exam? A: If the resident must be away, it is advised that you quit the program. The timer will pause while the program is shut down. When the resident returns you will need to enter in the proctor code. There is no penalty for closing the program. Q: Will it be up to the particular sites as to how many breaks will be issued during the exam? If so, is it best for all the residents to break at the same time? A: Yes, it is the purview of the program director proctoring the examination to decide how long the residents should break during test administration. Regarding breaks, your program should continue to follow the same routine you had with the paper version of the examination. Q: How much time will be allotted for the resident to complete the exam? A: 7 hours. The timer begins when the question 1 is answered. Q: What happens if the resident runs out of time? A: The resident cannot do anything but confirm their answers; the program will then lock the answers in the database and create the data file to the desktop. Q: Is there some sort of countdown timer on the CD? A: Yes. Once the resident begins the examination, a timer will begin counting down from seven hours. After the 7 hours the examination will shut down. Q: Is there going to be any technical support for residents not taking the exam on Saturday? A: There is always Technical Support during regular business hours at the Academy, Monday through Friday, 8:00 AM 6 PM CST. There will be Technical Support the day of the examination on Saturday, November 13, 2010 from 6:00 AM to 6:00 PM CST. Q: We were planning on scheduling some of the residents who will be on call on November 13 to take it on Friday, November 12. Is this a problem? A: On call residents are a special circumstance which the Academy understands; the OITE Administration Guidelines and OITE Scoring Policy allow under the purview of the Program Director that the examination be completed on Friday or up to Monday. Q: Any chance you could please confirm how long the actual exam goes for? Is it 5 or 7 hours? Does this include breaks? A: There are 7 hours allotted to complete the OITE. The software program has a built in timer. If the examination is shut down for approved breaks, the timer will pause and no time penalty will exist. Frequently Asked Questions /OITE DVD; 2010/10 Page 4 of 11

6 Q: How do we insure this is a closed book examination? Our university no longer has a computer room as all medical students are required to have laptops. All our residents will be using their own laptops that all have wireless connections and textbooks loaded on their hard drives. Although I do not anticipate this being an issue has this been considered? A: The software was developed so that residents could use their own laptops. The security features were programmed for the Resident Track and not the Practitioner Track. If a resident tries to access the web or any other application, the software will shut down and require the proctor code to restart the program. Answer Files Q: How are the scores/answers recorded? Do they save to the disc and send it in? A: The responses are saved two ways. They are saved in an encrypted file saved to the computer s hard drive. In addition, they are saved in an xml file which will be saved to the desktop. The proctor will copy that small xml file to a memory stick and send it as an attachment to the Academy. You can attach all the response files from all your residents and send them in one . The address is [email protected]. Q: Does the AAOS supply a memory stick? A: No. You will need a memory stick to collect the answer files from each resident s computer. Q: When should we the resident answer files? A: As soon as you have collected all the answer files you should them to the Academy at [email protected]. New for the 2010 OITE, the signed OITE Completion Form must also accompany your answer files. I recommend that you copy your Program Director in the . Q: When will we know that the answers files are safely at the Academy? A: Staff will verify that your files have been received and that the number of files matches those identified on your OITE Completion Form; however, it will be a few days before staff has the opportunity to verify all the individual answer files. It is important that programs keep a copy of the resident answer files until your score reports have been received. Q: How much memory will be needed on the memory stick in order to collect the resident s OITE question responses? A: A 2G memory stick is sufficient. These are very small files, approximately 5 KB. Q: What is the deadline for sending the resident answer files? A: The answer files must be received at the Academy by midnight CST, Monday, November 15, Q: What happens to the resident s answers if they are not using their own laptop? A: After the resident has completed the examination, locked in their answers, and closed the program, the proctor will use a memory stick to retrieve and save the answer file (*.xml). Then the resident can save the 2010 OITE Program folder to their own separate memory stick. This Frequently Asked Questions /OITE DVD; 2010/10 Page 5 of 11

7 folder is located on the hard drive of the computer. In PC format the folder can be found in the following location: C:\Program Files\2010 OITE. Then the resident can copy the folder onto their own computer so that when the review mode code is released, they will be able to compare their examination responses with the preferred responses. In addition, any notes written to the note pad feature are contained in the same folder. Q: How does the proctor collect the answer files? A: Upon completion, a confirmation page will display on the resident s computer. The software will ask the resident to confirm their answers. When the resident s confirms their answers, an answer file will be saved to the desktop (name.xml). The file is transferred to a memory stick by the proctor. Q: How do I send the answer files to the AAOS? A: Once all the data files have been saved to the proctor s memory stick; the proctor will send an to [email protected] and attach all the files to the . In addition, the signed OITE Completion Form must accompany the answer files. Q: Will there be more than one scoring? A: No, there will only be one scoring. Q: Will you know when the resident took the examination? A: Yes, unlike the paper answer sheets, the electronic files will have the date & time that the answers were confirmed built into the answer file. Q: Will you know how many times the computer shuts down? Is there a penalty for too many shut downs? A: We will know how many times the computer shut down from the electronic answer file. This is a new feature and no decision has been made regarding the number of shut downs that should be within an acceptable range. To-date, there is no penalty for computer shutdowns. Q: What would happen if the resident did not copy the user.rbd file to their computer? A: If the resident used a residency or hospital computer to complete the OITE and did not save a copy of the [2010 OITE] folder to copy to their personal computer they would not see their own answer choices when they open the OITE in the Review Mode. After the access code for the Review Mode was entered into the system the examination would open, but only the preferred response would be visible. The resident answer selections and any notes created on the name of the examination would not be available. Q: Since the OITE is electronic will we get our results sooner? Will the residents receive the review code directly from the Academy? A: Per section 3.3 of the OITE Scoring Policy, programs can expect posting of the OITE Score Reports to the Program Director s Profile Page on the AAOS website within 8 weeks of the examination date. Preparation Frequently Asked Questions /OITE DVD; 2010/10 Page 6 of 11

8 Q: Should I be forwarding these chapters on to our residents or is this just for me? A: These chapters were developed for the proctors; however, you may distribute these chapters to the residents or other individuals that you think should read them. Q: Is it okay to post the Chapters on the resident website? A: Yes. You may distribute these chapters to the residents. Q: Do we have to read all the User Manual chapters? A: No, the chapters were created to provide programs with sufficient information for a successful examination day using a new technology. The chapters are optional. Q: When will the discs be sent out for the residents? A: The disc will be sent a week ahead of time similar to the books. Q: If we install the OITE CD-ROM onto the resident s computer beforehand, will they have access to the exam before the November 13 test date? A: You or your IT staff can install the software prior to November 13 th, which is recommended and then close the program and remove the disc from the CD-ROM drive. You must remove the disk after the software has been installed and keep it in your possession until the day of the examination. As long as the disc is not in the computer drive, there is no access to the actual examination. Q: If our residents are using individual laptop computers, should we purchase additional power strips to avoid low battery interruptions during the exam? A: It is always recommended for any computer that they be on a surge protector to protect the computer from a surge. Seven hours are allotted to complete the examination; residents will need to be aware of their laptop s battery constraints, if electric power is not available. Q: What happens if a laptop is used and the battery dies; will the resident be allowed to plug in electrically and continue with the test? What happens if power to the computer is disrupted; will the resident be able to continue the test after the power is restored? A: If the power is interrupted the examination will shut down. You can plug in the computer and go to the desktop to log in. If the time has not expired, then you will use the proctor code to restart the examination. The answer files are written to an encrypted folder on the hard drive so no information will be lost. Q: How can my residents and faculty view the chapters of this manual? A: The chapters are also posted on our Technical Support website under the 2010 OITE link which can be found at Q: Do the residents have to complete the evaluation? A. No, the purpose of the evaluation is to collect information regarding what was positive and what was negative in the program so that the Academy can make improvements. Q: Maybe this has been answered, reading the manual for the second time when will the software be sent so I can have our IT folks to load the software on the computers? Frequently Asked Questions /OITE DVD; 2010/10 Page 7 of 11

9 A: The examination discs will be shipped the first week of November. They will be sent Federal Express. The OITE CDs are scheduled to arrive at your program on Monday; we request all coordinators to complete the OITE Shipping Survey on the SharePoint site as soon as you receive your shipment. Please open the box and verify the contents. The IT people can use one disc to load all the computers before the test. It will take less than five minutes to install the software on each computer. Review Mode Q: When will we receive the Review Code? A: The Academy will release the review code when we have all the data from all the programs. The most likely release date for the review code is 4 weeks after the examination date. Q: What happens when the residents receive the Review Code? A: The software will ask for the review code after completing the examination similar to how the proctor code is entered. Once that review code is entered, the software can be opened so that the user can see the recommend readings and the preferred response in comparison to their answer choice for each examination question. Q: Can several residents use the same computer to get into Review Mode? A: Only one examination can be on a single computer. Anyone can access that examination in review mode that sits on the computer but only the responses that belong to that resident s examination will display. Frequently Asked Questions /OITE DVD; 2010/10 Page 8 of 11

10 Alternative Arrangements Q: Will there be Technical Support on Monday for our residents who are on call? A: Yes, there will be Technical Support on Saturday, November 13, from 6 AM to 6 PM CST. There will be no technical support on Sunday. There will be technical support on Monday during regular business hours. Q: If a resident is on call, can he/she take the exam at a later date. A: Yes, this is outlined in the OITE Scoring Policy and OITE Administration Guidelines. Q: What is the procedure if an on-call resident takes the exam on Sunday or Monday? A: The procedure is the same as on the examination day. You may your answer files on the day of the examination with the majority of your residents and send the remaining file when they complete the examination or you may wait till you have all your residents answer files. Q: If a resident takes the exam on Saturday on a university owned PC, will he/she be able to see answers on another personal computer, e.g. a Macintosh? A: Yes, but only if the residents copies the 2010 folder that has been saved to the hard drive and then copies that folder to their personal computer. The software must have the encrypted user.rbd file of the resident saved on the computer in order for the review mode to show their responses. Q: We were planning on scheduling some of the residents who will be on call on November 13 to take the examination on Friday, November 12. Is this a problem? A: See the OITE Scoring Policy and OITE Administration Guidelines. Q: What if I have a resident who is on vacation during the OITE, is there a make-up date? A: No, there are no make-up dates. See the OITE Scoring Policy and OITE Administration Guidelines for other options. Q: We have 52 residents taking the exam. Our computer lab has only 20 stations so we had to spread the test to 3 days: Is this okay? A: Having the residents take the examination over a period of three days is not optimal. Many programs are using a combination of personal and program computers. After the first group of residents completes their examination, it is no longer confidential. If however, the residents must take the examination in more than one setting; you must retrieve the examination disc back from each resident that has completed the examination, until every resident has completed the OITE. Then you can re-distribute the discs. Q: Can one resident be substituted for another resident on the day of the exam? A: Yes, the new resident will put their information in the registration page of the examination. When sending the answer files, include this information. Q: If a resident is going to be at another location at the time of the exam, what are the necessary procedures to have the exam delivered to an alternate site? Frequently Asked Questions /OITE DVD; 2010/10 Page 9 of 11

11 A: The resident can take the examination at another location similar to when there was a print book. The resident should fill in their program number using the registration page that is provided on the package sticker. Q: How do we administer the OITE if the resident is assigned to a rotation at an external site? Since examination will be done through computer now, the institution wants to know the procedure. Do we just send the CD and the instructions and they can install the program in their system? A: Arrangements should be made at the alternative site. A disc could be sent to the individual who has agreed to proctor the examination of that resident. The User Manual chapters can be made available to both the resident and the proctor. The proctor at the external site should be made aware that the OITE is not an on-line examination and that the software has security features which will cause the program to shut down if the resident tries to access any other program while in test taking mode. The proctor will have to enter the pass code to restart the examination. Q: What if the resident was in the middle of taking their OITE and they were called out for a patient, and could not return that day? A: This is a significant change because if the software is shutdown beyond the calendar date the examination was started, then the examination would act as if the 7 hours were up and the resident s answer file would be exported incomplete. Therefore, if this happens at your program on the day of the examination you will need to complete the following steps. (Example is using the practice disc) Step One: Shut the program down. Keep a copy of the residents answer sheet with their answers and their OITE Disc. Frequently Asked Questions /OITE DVD; 2010/10 Page 10 of 11

12 Step Two: If the resident returns in the same calendar day and has sufficient time to complete the examination, and then just begin the OITE as any other break. Step Three: If the resident was unable to return within the same calendar day, then the software should be uninstalled from the computer and re-installed. The resident would then begin the examination from the beginning, and would enter their responses from their answer sheet and continue on. All other procedures would remain the same. Practitioner Discs Q: What will the deadline be for faculty to take and return their OITE answers? A: Faculty have until April 1, 2011 to return their answer files. Faculty will send an to and attach their answer file. As soon as their answer file is received by the AAOS for scoring, practitioners will receive an access code to open their examination in review mode. Q: Can faculty still order an OITE? A: Yes, Practitioners can order an exam through March 14, 2011 and their completed answer file to the Academy by April 1, 2011 to receive CME credit. Frequently Asked Questions /OITE DVD; 2010/10 Page 11 of 11

13 OITE DVD-ROM TUTORIAL Tutorial/OITE: of 4

14 OITE DVD-ROM TUTORIAL 2 2 of 4

15 OITE DVD-ROM TUTORIAL 3 3 of 4

16 OITE DVD-ROM TUTORIAL 4 4 of 4

17 OITE User Guide

18 Program Interface When you begin the exam, you will see the screen above. The interface consists of a Top Menu bar, a question field, 5 answer choices, image thumbnails, navigation buttons and a Mark button. Figures Questions may contain thumbnails for Figures referenced in the question. Single-click on a thumbnail to view the Figure larger.

19 You can also open multiple Figures at a time. PLEASE NOTE: When you click on multiple thumbnails at a time, the previously opened Figure will move behind the question screen. To view all the Figures on screen at the same time, minimize the Question Screen by clicking on the Minimize button located in the upper right corner, above the AAOS logo: After minimizing the Question Screen to the bottom left corner, you can resize the Figures by single-clicking on the bottom right corner and dragging your cursor inwards. Move the Figures around so you can compare them. When you are done viewing the Figures, make sure to close them first before restoring the Question Screen. Restore the Question Screen by clicking on the Restore button shown below: MAC USERS: The MAC OS will not open the program in a main window as shown above. All windows will be free floating.

20 Select an Answer Click on an answer choice to make your selection. A blue highlight will indicate your selected answer. You can return to a question to change your answer at any time before the end of the exam. You can review all answered questions. Click Views in the Top Menu bar and select Answered Questions.

21 The Question Screen will display all the questions you have answered as shown below: Skipping a Question You can skip a question and come back to it later to answer it. When you click on the Next button and do not click on an answer choice, the question is considered skipped. To display questions you have skipped, click on Views in the Top Menu bar and select Skipped Questions. The Question Screen will display all the questions you have skipped as shown below:

22 Marking a Question You can mark a question for later review by clicking on the Mark button on the screen. This will change the text on the button to read Unmark. To unmark a question, simply click on the Unmark button. To display questions you have marked, click Views in the Top Menu bar and select Marked Questions. The Question Screen will display all the questions you have marked as shown below: PLEASE NOTE: Questions marked during the exam will remain marked when you access the Review Mode of the program.

23 Jump To Feature To jump directly to a specific question, click Views in the Top Menu bar and select Jump to a Question. You must enter a question number in the window or click Cancel to close it. Notes Feature The program includes a Notes feature that allows you to add comments about a question. Click on Tools in the Top Menu bar and select Notes.

24 A window will open for you to type in and save your notes. When you are done typing your notes, click on the Finished button. An icon will appear on the question screen if you have saved a note. You can read the note by single clicking on the icon. You can review questions you made notes for. Click Views in the Top Menu bar and select Questions with Notes.

25 The Question Screen will display all the questions you have saved noted on as shown below: Confirmation Screen The Confirmation Screen can be accessed at any time. Click Views in the Top Menu bar and select Confirmation Screen. You are given a status on Answered, Skipped, and Marked questions. To view these, click on the View button next to each category.

26 Once you have answered all the questions, the Confirm Answers button will become active for you to lock all of your answers for submittal to AAOS. After clicking the Confirm Answers button, the program will open a survey with questions, followed by the Export Data Screen. Export Data Screen Practitioners will see the export screen on the left with specific instructions on how to submit for CME while Residents will see the export screen on the right with instructions on submitting their data.

27 Review Mode Access Code Window When you have finished the exam and restart the program, you will be prompted for a Review Mode Access code: Once your data has been submitted to AAOS, you will receive the code via . Enter the code and click the Accept button to access the program s Review Mode. Also found on this window are the Exam Print Copy URL button and the Re-Export Data button. The Exam Print Copy URL button will open your web browser and take you to the AAOS website where you will be able to download a version of the exam for you to review while you wait for the access code. You must have an internet connection to access the AAOS site. The Re-Export Data button will export your exam data again should you need to resubmit it to AAOS. PLEASE NOTE: Once you enter the code, this window will no longer appear. You will still be able to view the PDF version of the exam in Review Mode but you will not be able to re-export your data. OITE Review Mode Review Mode will unlock the following features: Answer Selection Highlights Recommended Readings Content Domains Marked Questions Print Function

28 Answer Selection Highlights The answer you chose will be highlighted in blue. The 'preferred' answer will be highlighted in orange. If you only see your answer selection highlighted in blue, that means you selected the 'preferred' answer. If you left any questions unanswered, only the 'preferred' answer will be highlighted. Recommended Readings Each question screen will have a Recommended Readings button that displays a list of references pertaining to that question. You can access this by clicking on the Recommended Readings button at the bottom left of the interface: You can also access Recommended Readings by clicking Tools in the top menu and clicking on the Recommended Readings link: When you click on either link, a window will open with the list of recommended readings for the question you are viewing:

29 Content Domains Questions will be sorted by 12 domain types. Click Views in the Top Menu bar and select Content Domains. Select from the list of 12 domain types to view all questions relating to each domain. Marked Questions Questions you left marked during the exam will remain marked to facilitate your review. Print Function Your Notes and a PDF version of each question are printable during Review Mode. When you open notes you saved during the exam or during Review Mode, you will notice a Print button in the window: Clicking on the Print button will print out your notes along with the question, answer choices, preferred answer and the Recommended Readings for that question screen.

30 You can access the PDF version of the exam again by clicking on Tools and then clicking on Exam Print Copy URL. This will open your web browser and take you to the AAOS website where you will be able to download a PDF version of the exam which will allow you to print out the questions, answer choices, preferred answer, recommended reading and images. PLEASE NOTE: You must have an internet connection to use the feature.