Applied Science Laboratories. Model 504. Eye Tracker and Gaze Tracker. System Setup and Operations Manual

Transcription

1 Applied Science Laboratories Model 504 Eye Tracker and Gaze Tracker System Setup and Operations Manual Original C by Applied Science Group, Manual version 2.4 Revisions C by Michael Wogan, Ph.D., J.D., Department of Psychology, Rutgers University, Camden, N.J All rights reserved. Applied Science Laboratories 175 Middlesex Turnpike Bedford, MA Phone: (781) Fax: (781)

2 Table of Contents 1. Introduction General System Description Major Assemblies Pan-tilt eye camera optics module Floor Mounted Scene Camera or Scan Converter Control Unit Monitors Interface PC Software Interface Description Options Binocular and High Speed Magnetic Head Tracking Hardware Installation Unpacking and Assembly Control Unit Pan Tilt Optics Scene Camera Scan Converter Eye Tracker Interface PC Magnetic Head Tracker Component Placement Optics and Scene Magnetic Head Tracker Placement

3 3.3 Connections Pan-tilt Optics Module Wide Angle Locating Camera Obtaining an Eye Image Eye Tracker Interface PC Black and White Monitors Magnetic Head Tracker Direct Use of Scene Video Software Installation Eye Tracker Interface Software Uploading Program Information to the Control Unit E5000 User Interface Program User Interface Screen System Settings Saving and Retrieving Default Data Enabling Magnetic Head Tracker Eye Tracker Calibration Setting Up Calibration Target Points Overview Set Calibration Target Points Check Calibration Target Points Calibration of Magnetic Head Tracker Getting Started MHT Transmitter to Pan-Tilt Calibration MHT Sensor to Pan-Tilt Calibration

4 5.2.4 Checking the Calibration Procedure Subject Setup and Calibration Subject Seating Obtaining an Eye Image Aiming the Camera Focusing the Camera Enabling the MHT System and Setting Sensor-to-Eye Offset Pupil and CR Discrimination Controlling Ambient Illumination Controlling Monitor Brightness Controlling the Brightness of Stimulus Materials Subject Calibration Nine Point Calibration Custom Calibration Options Two-point Calibration Seventeen Point Calibration Option Manual Eye Position Offsets Saving and Loading Calibration Files Eye Movement Monitoring Blink Handling Data Recording Open a New Data File Start Recording Event Markers XDAT External Data

5 7.2.5 Auto Eyedat Stop Recording Close Data File View Recorded Data Working with EYEDAT files Vertical and Horizontal Eye Position Coordinates Conversion of Pupil Diameter to Metric Units Magnetic Head Tracker Data in X-Y-Z Dimensions The Sample as a Sliding Average An Example of Data Output Serial Data Output Interface Cable Protocol and Data Format Analog Data Output Theory of Operation and Specifications Pupil and CR Recognition Eye Line of Gaze Computation Timing Specifications Trouble Shooting General Approach Functional System Description Functional Priorities Preliminary Trouble Shooting Trouble Shooting Tips

6 9.5.1 Specular Reflection (glasses) Eye Calibration and Pupil/CR Recognition Target Points MHT Using the model eye Gaze Tracker Setup Hardware Connections Installing the Gaze Tracker Program Synchronizing Gaze Tracker with Eye Tracker Testing the Accuracy of the ET to GT Data Transform Fine tuning the ET/GT Calibration Sanity Checks Creating a Slide Show Calibration Slide Interspersed Text Slides Saving the Slide Show Editing the Configuration Creating a New.mdb File Recording and Saving Data Recording Saving Data Checking Saved Data Look Zones Creating Look Zones Editing and Manipulating Look Zones

7 13.3 Saving Look Zones Data Analysis Using Gaze Tracker Analysis/ Data/ Export: Outputting Raw Data and Summary Statistics (text file) Text File Output: X-Y Sample Data Text File Output: Fixation Data Exporting a Summary for Multiple Subjects (Excel file) Working Around Program Limitations Appendix 1. Startup Instructions A1.1 Short Startup Checklist A1.2 Startup Checklist (longer version) A1.2.1 For this Session A1.2.2 For this Subject Appendix 2. Program Menus and Buttons A2.1 Eye Tracker Menus A2.2 Eye Tracker Buttons A2.3 Eyenal Menus A2.4 Fixplot Menus A2.5 Gaze Tracker Menus A2.6 Gaze Tracker Buttons

8 Table of Figures Figure 2-1. Model 504 Eye Tracking System with optional Magnetic Head Tracker Figure 2-2. Model 504 with Scan Converter Figure 2-3. Standard Components of Model 504 Eye Tracker Figure 2-4. Pan-tilt Camera Optics Module Figure 2-5. Pan-tilt Module with Tilt Knob Figure 2-6. Front of Pan-Tilt Camera Figure 2-7. Model 5000 Control Unit Front Panel Figure 2-8. Model 5000 Control Unit Rear Panel Figure 3-1. MHT sensor marked to show up (Y+) Figure 3-2. Measurable Field of View Figure 3-3. Measurable Field of View from Subject s Point of View Figure 3-4. Optimal placement for Camera and Monitor Figure 3-5. Computation of visual angle between points on a flat surface Figure 3-6. Camera placed beside Monitor Figure 3-7. Camera placed in front of a very large Monitor Figure 3-8. Camera Mounted on Tripod in Front of Projection Screen Figure 3-9. Placement of Magnetic Transmitter Figure Magnetic Transmitter mounted on Wooden Post Figure Connections for Model 504 Eye Tracker with Magnetic Head Tracker Figure Connections using Scan Converter to supply Scene Video signal Figure Connections using Scene Video Signal in place of Scene Camera Figure 4-1. Eye Tracker Main Screen with Upload Dialog Box Figure 4-2. Eye Tracker Main Screen - Upload Complete Figure 4-3. System Settings Dialog Window Figure 4-4. MHT Pull-down menu Figure 4-5. Selecting MHT Type Figure 4-6. Interface Program Main Screen showing MHT Transmitter Offset Figure 4-7. MHT Sensor Orientation Figure 5-1. Nine-point Calibration Target Figure 5-2. Set Target Points Shortcut Button Figure 5-3. Set Target Points Window Figure 5-4. Setting Target Points Figure 5-5. Check Target Points Shortcut Button Figure 5-6. Check Target Points Window Figure 5-7. Pan-Tilt Setup Window Figure 5-8. Confirm MHT Calibration Figure 5-9. Measuring Distance from Center of Transmitter to Pan-Tilt Camera Center of Rotation Figure Set Camera to Transmitter Distance Figure Place Sensor between Camera and Transmitter Figure Continue MHT Calibration? Figure Place Sensor to Left Figure MHT Sensor Calibration Shortcut Button Figure 6-1. Controlling Camera Focal Length

9 Figure 6-2. Set Sensor-to-Eye Vector Button Figure 6-3. Setting Threshold Discrimination for Pupil and Corneal Reflection Figure 6-4. Pupil Outline and Corneal Reflection as seen in Eye Monitor Figure 6-5. Eye Monitor View showing Stable Pupil and CR Discrimination Figure 6-6. Speckling or Flaring around the Eye Figure 6-7. Eye Calibration Shortcut Button Figure 6-8. Eye Calibration Window Figure 6-9. Custom Eye Calibration Window Figure Seventeen point Calibration Target Figure Eye Position Offset Figure 7-1. Recording from Eye Tracker Main Screen Figure 8-1. Relation between Line of Gaze and Pupil/CR Separation Figure 8-2. Eye Tracker Timing Figure Gaze Tracker Opening Screen Figure Choose an Analysis Mode Figure Load a Slide Show Figure Gaze Tracker Main Screen Figure Toggle Slide View Figure Data Transform Window Figure Test Connection Figure Slide Show Wizard Step One Figure Adding Slides to a Slide Show Figure Setting Slide Presentation Parameters Figure Advance Slide when Mouse Click is Detected Figure Advance Slide after 10 Seconds Figure Mouse Click or Space Bar will Advance to the Next Slide Figure Completed Slide Show Figure Saving the Slide Show Figure Configuration Notice Figure Warning Message Figure The Manage Database Window Figure Editing a Single Subject Figure Advance to the Next Slide Figure Save Data Panel Figure Assigning Subject ID Number Figure Subject Identifying Information Figure Save Data Before Exit Prompt Figure Checking Saved Data Figure Recorded Sample Points for One Slide Figure Apply Look Zones Box Figure Look Zone Properties Dialog Box Figure Look Zone Data Dialog Box Figure Save Look Zone Data

10 Figure Choosing Defaults for Fixation Options from the Gaze Trail Options Window Figure Selection of Data Type Figure Summary Data for Look Zones Figure Summary Data Options Figure Slide Selection Figure Selecting Look Zones Figure Subject Selection Figure Summary Figure Three Factor Excel Data File Figure Two Factor Excel Data File

11 CHAPTER 1: INTRODUCTION The Model 504 remote eye tracker is designed to measure a person s pupil diameter and point of gaze on a stationary (fixed) scene. The point of gaze (POG) is displayed as a cursor, a cross, or a set of cross hairs superimposed on the image the subject is viewing. The image can come from a scene camera or other video source, such as a monitor. If the data is being recorded by the Eye Tracker program, the data file is stored on the Interface PC. If the data is being recorded by an external program, such as the Gaze Tracker program, the data file is stored on the Display PC. The data may also be exported as a real time serial data stream to an external device. Recording on the Interface PC is described in Chapter 7. Real time serial data output is described in Section 7.5. Instrument technical specifications are summarized in Section 8.4. The auto-focus eye camera and the eye illuminator (IR beam) are contained in a pan-tilt module that automatically moves the camera and illuminator in both azimuth and elevation to follow motions of a subject s head. The Magnetic Head Tracker system tracks the position of the subject s head, including head rotation, within a limited field by means of a small magnet held in place by a velcro band. The Interface PC running the Eye Tracker program serves as the experimenter s interface with the eye tracker system. The Eye Tracker program stores calibration information in a special file. When properly calibrated, the Eye Tracker, operating through the Control unit, controls camera tracking and calculates the subject s Point of Gaze and pupil diameter at each sample point. The Eye Tracker may also be used to record this data. A general system description is provided in Chapter 2, followed by unpacking and installation instructions in Chapter 3. The Eye Tracker program is described in Chapter 4. System calibration and operation are described in Chapter 5. Chapter 6 describes the process of subject calibration, which must be done for each subject. Data recording with Eye Tracker is described in Chapter 7. The basic theory of operation is explained in Chapter 8. Chapter 9 will be useful for fault diagnosis should there be any problem with system function. A variety of options are available to enhance performance of the 504 system. Some of the options are described in section 2.3. The Gaze Tracker program runs on the Display PC. It controls the stimulus display and timing, and data recording and output. Point of gaze information is passed to Gaze Tracker from the Eye Tracker Controller via a serial connection. Chapter 10 describes installation and calibration of the Gaze Tracker program. Chapter 11 describes the process of creating a new configuration or slide show. Recording and saving data is described in Chapter 12. Once the data has been recorded, the gaze path can be played back in real time, superimposed over the stimulus display. What Eye Tracker calls an Area of Interest (AoI), Gaze Tracker refers to as a Look Zone (LZ). These are simply zones of particular interest in the stimulus display, as defined by the user. Creating Look Zones is described in Chapter 13. Once Look Zones have been created and saved, they may be displayed or hidden in the displayed image. Data analysis and export is described in Chapter 14. The Gaze Tracker manual, contained in a.pdf file on the Display Computer in C:\ERT\bin\manuals\>, gives additional information about creating 3-D displays, and

12 so forth. Appendix A provides a checklist of steps to be followed each time a subject is run. Appendix B gives a list of Eye Tracker menus and short-cut buttons, and Gaze Tracker menus and short-cut buttons. Following the usual Windows convention, once the program is running, moving the cursor over a short-cut button causes the button s label to appear.

13 Chapter 2: General System Description The Model 504 eye tracker with an optional magnetic head tracker (MHT) is illustrated schematically in Figure 2-1. The subject s eye is illuminated by an infrared beam on the pan-tilt camera optics module. An auto-focusing lens system in the pan-tilt camera focuses an extreme close-up (telephoto) image of the eye. The pan-tilt mechanism can rotate both the illumination source and eye imaging components in azimuth (pan) and elevation (tilt) in order to follow the eye as the subject or the eye moves. A second, wide angle lens in the pan-tilt camera moves in parallel with the telephoto lens to track the general orientation of the camera. A separate scene camera may be used to record the scene being viewed by the subject. Both the pan-tilt camera and the scene camera, if used, are connected to the model 5000-control unit. Two black and white monitors, Figure 2-1. Model 504 Eye Tracking System with optional Magnetic Head Tracker supplied by ASL, are used to monitor the system. An alternate configuration, shown in Figure 2-2, assumes that the subject will be viewing a computer monitor. A scan converter is used in place of the scene camera to produce a video copy of the display viewed by the subject.

14 Figure 2-2. Model 504 with Scan Converter The Interface PC runs the Eye Tracker program and controls the operation of the system. The Eye Tracker program is used to calibrate both the system and the individual subject. The Display PC runs the Gaze Tracker program and controls the display shown to the subject. The Eye Tracker collects eye movement data and passes it on to the Gaze Tracker program, where it is stored. The Model 5000 Control Unit processes the image from the close-up lens to extract the elements of interest (pupil and reflection of the light source on the cornea) and computes both pupil diameter and point of gaze. These data are displayed and output to external data ports. For a discussion of the principles used to determine eye line of gaze see Chapter 7, Theory of Operation. The Control Unit uses pupil recognition information to control the pan-tilt camera, determining appropriate pan and tilt commands to keep the pupil image centered. The magnetic head tracker provides information about head rotation or movement even when the eyes are closed or out of the camera field of view. When this occurs, the Control Unit uses information from the magnetic head tracker (Figure 2-1) to swiftly re-position and re-focus the camera when the eye comes back into the field of view. A hand-held remote control unit is used by the operator to manually control the camera and adjust the focus. The hand-held control unit must be used for initial calibration of each

15 subject. The black and white Scene Monitor has an A/B switch to control the input source. In one position, the monitor displays the view from the pan-tilt camera wide angle lens, to permit tracking and aiming of the camera. In the other position, the Scene Monitor shows the display as seen by the subject. Once a stable signal is acquired by the software, pupil and corneal reflection outlines and center cross hairs are displayed in the Eye Monitor over the close-up image of the eye. If the signal for either pupil or CR is lost, the outline and cross hairs disappear and no valid data is being recorded. The point of gaze (POG) is calculated and displayed as a cursor or set of cross hairs superimposed on the scene in the Scene Monitor. The X-Y position of the eye is also displayed in the Scene POG window in the Eye Tracker main screen (see Chapter 5, Section 5.3 through 5.5). Calibration commands and most other interaction with the operator take place through the Interface PC. Data may be recorded on the Interface PC hard disk and processed later by user-written programs or by ASL s data analysis (EYENAL) program. For systems using the Gaze Tracker program, the Interface PC passes the data to the Display PC. The data is recorded on the Display PC by the Gaze Tracker program. Setting up communication between Gaze Tracker and Eye Tracker, and use of the Gaze Tracker program are described in Chapter 12. The standard model 504 consists of the items listed below: 1. Model 5000 Control Unit and plug in power supply 2. Pan-tilt optics eye camera optics module. 3. Scene camera with tripod; or scan converter 4. Two b/w video monitors (one for the eye image and one for the scene image) 5. Cables for connecting the Model 5000 Control Unit to the Interface PC, to the pan-tilt optics and scene camera, and to the two monitors. A minimum system configuration should include all of these items, plus two PC s supplied by the user, with minimum specifications as described in Section Figure 2-3 shows the standard model 504 components.

16 Figure 2-3. Standard Components of Model 504 Eye Tracker Note that a scan converter is substituted for the scene camera if subjects will be viewing a computer monitor display. 2.1 Major Assemblies Pan-tilt eye camera optics module The pan-tilt camera shown in figure 2-3 and 2-4 normally rests on a horizontal surface, such as a table top, but it can be mounted on a tripod. A ring of near infrared LED s surrounding the close-up lens provides eye illumination. The power switch and connectors are located on the rear of the camera base, except for the illuminator power connector (6 vdc, 100 ma), which comes out of the rear of the camera. A separate power supply (13.5 vdc) connects to a DC power input connector on the rear panel. The transformer plugs into a standard AC wall outlet. A hand-held IR remote control unit provides manual control for pan-tilt positioning, lens focus and zoom, and various other features built into the pan-tilt camera. The camera video output is 60 Hz (NTSC) or 50 Hz (PAL) composite video format depending upon the video standard in the destination country.

17 Figure 2-4. Pan-Tilt Camera Optics Module. The pan-tilt mechanism provides about 100/ of pan angle and 25/ of tilt capability. It may be necessary to tilt the base of the unit to achieve the necessary camera view angle. A removable knob is provided for this purpose. The tilt knob screws into a ¼ - 20 hole in the unit base plate and, when in place, can be adjusted to hold the front of the base plate between 1.25 and 1.5 inches above the table surface (see Figure 2-5). The same mounting hole can be used to fasten the unit to a tripod. The camera has an auto-focusing close-up lens with a zoom function (Figure 2-6). The zoom setting controls the range over which the camera can be focused. At maximum zoom, the camera will auto focus over a range from about 18 to 24 inches. As zoom decreases, the width of the field of focus increases. Ordinarily, the lens need not be used at full zoom. The maximum distance at which the eye tracker can be used effectively is about 40 inches from the camera to the eye (see Figure 2.5).

18 Figure 2-5. Pan-tile Module with Tilt Knob The camera also provides a separate wide-angle lens with an axis parallel to the axis of the zoom lens (see Figure 2-6). The view through the wide-angle lens appears in the Scene Monitor when the A/B switch is switched to the A position. The wide-angle lens and the handheld remote control are used during calibration to aim the camera and find the subject s eye (see Section 3.3.2). Once calibration of the subject is complete, the A/B switch can be switched back to the B position. The Scene Monitor then shows the same scene as the one being viewed by the subject. Figure 2-6. Front of Pan-Tilt Camera. The remote control for the camera is an IR emitting device and must be pointed at the sensor on the front of the pan-tilt camera. If the operator is sitting behind the camera, it can be adjusted from the rear by bouncing the IR beam off of a mirror, positioned in front of the camera and angled so the IR beam is reflected into the sensor on the front of the camera. This

19 permits the operator to control the camera while watching the black and white monitors, which will be facing away from the subject. An amber light on the base on the front of the camera blinks when the camera receives a command from the hand-held control. If the light doesn t blink when the remote control is operated from a position facing the front of the camera, a new set of batteries should be installed and tested in the remote control. Except for auto-focus, pan-tilt camera functions can be controlled from the Eye Tracker Interface program, as described in Chapter 4, but the adjustments using software are slow and not as smooth as when the hand-held control is used. The hand-held remote control is an integral part of the eye tracker system. Keep a spare set of batteries. The Auto-focus function of the close-up lens can only be switched on/off with the handheld remote control. It cannot be controlled from software and there is no indicator, either in the Eye Tracker program or on the camera, which tells whether Auto-focus is on or off. The only way to tell whether or not the Auto-focus is on is to point the camera at a stationary object and attempt to refocus the lens. Watch the behavior of the lens focus in the close-up image in the Eye Monitor. If the auto-focus wants to fight and over-ride the manual control, Auto-focus is On. Once the subject s pupil is clearly focused in the frame during calibration, Auto-focus should be switched to on Floor Mounted Scene Camera (FMSC) or Scan Converter A scene camera may be used to produce a video image of the same scene the subject views. This scene image provides the reference frame for the eye point of gaze measurement. A color scene camera (not provided by ASL) may be powered directly from the model 5000 Control Unit. If used, it is equipped with an 8 mm lens, and includes a standard photography tripod to support the camera. The scene camera video output is 60 Hz (NTSC) or 50 Hz (PAL) composite video format depending upon the video standard in the destination country. If subjects will be looking at an image on a computer screen, the system includes a scan converter in place of the scene camera (see Figure 2-2). The scan converter converts the VGA computer screen image, as viewed by the subject, to a composite video signal (either NTSC or PAL standard) which is input to the eye tracker control unit as the scene video image. The scan converter is powered by its own external AC supply Model 5000 Eye Tracking System Control Unit The Model 5000 Control Unit measures 3.25 h x 10.0 w x d. It weighs 4.5 lb., and includes an external 12 VDC power supply. The unit front panel (Figure 2-7) contains a power switch; the rear panel (Figure 2-8) contains all necessary connectors. Basic use of the device requires the connectors labeled Controller, Camera, Remote Scene, Eye Out, Scene Out, and 12V DC IN. These are the connections, respectively, to the Interface PC; the eye camera; the scene camera or subject display monitor; the eye monitor; the scene monitor; and the DC power supply respectively. Other connectors support various options or secondary functions such as communications with a magnetic head tracker and data output to an external device. The HMO/PT slide switch should be in the P/T position for use with pantilt optics.

20 Figure 2-7. Model 5000 Control Unit Front Panel. Figure 2-8. Model 5000 Control Unit Rear Panel. The connections are summarized as follows: Controller - from Interface PC (COM 1) Camera - from eye-tracking camera Remote - from Scan Converter (subject display monitor) Eye Out - to Eye Monitor Scene Out - to Scene Monitor 12 VDC In - from power transformer The control unit houses the processing board that receives video input from eye and scene cameras, recognizes pupil and corneal reflection (CR) features in the video eye image, computes line of gaze, communicates with the Interface PC, controls the pan-tilt mechanism,

21 and superimposes feedback outlines, cross hairs, and cursors on the eye and scene video signals for monitor display. When an optional magnetic head tracker is used, the processing board also communicates with the magnetic head tracker (MHT) and uses the head position data from the MHT sensor to help control the pan-tilt mechanism Monitors As shown in Figure 2-2, the system includes two black and white video monitors. One is an Eye Monitor, which shows a close-up image of the pupil, the second is a Scene Monitor, which shows the scene being viewed by the subject. The Scene Monitor is equipped with an A/B switch. In the B position, the Scene Monitor shows the view through the wide-angle (longrange) lens on the pan-tilt camera; in the A position, the Scene Monitor shows the scene as viewed by the subject. During successful data collection, cross-hairs indicating the Point of Gaze will be superimposed on the scene or stimulus frame. The Eye Monitor displays the image from the camera close-up lens. When the eye tracker is functioning properly, a white outline is superimposed over the image recognized as the pupil and a black outline is superimposed on the image recognized as the corneal reflection (CR). A white set of cross hairs designates the center of the pupil and a black set designates the CR. The white outline of the pupil is offset slightly from the pupil for greater visibility in the display. If monitors are supplied by the user, video cables must also be supplied. The monitors may be black and white or color and must accept a standard EIA (black and white) or NTSC (color) composite video signal (CCIR or PAL if 50 Hz cameras are being used) Eye Tracker Interface PC A computer may be supplied by ASL to serve as the Interface PC, but is usually supplied by the user. The computer serves as the user interface device and as a datarecording device. ASL always supplies an Eye Tracker Interface program, which runs on this computer. System requirements for the Interface PC are an IBM compatible PC capable of operation with Windows 95, Windows 98, Windows NT, 2000, XP, or DOS. A computer that can run only DOS is limited to using only the DOS version of ASL s Eye Tracker Interface Program. The computer must have COM1 or COM2 serial ports available using standard interrupts and device addresses. The minimum recommended system is a 200 MHz Pentium, but slower systems will work adequately as well. In these cases consult ASL Software The Eye Tracker software is run from the file E5Win.exe, which will be on the Interface PC in the C:\ProgramFiles\ASL Eye Tracker\EyeTracking\> directory once the program has installed itself (spaces in the path name are intentional). The EYEPOS software package necessary to operate the eye tracker and record data is provided as a standard part of the Eye Tracker. The EYEPOS software is included on a CD ROM, labeled EyeTracker Software, or on a floppy disk labeled EYEPOS. Also included on the CD ROM or on a set of floppy disks is the EYENAL data analysis software, which includes a CONVERT function to convert binary data files to ASCII, and ACCESS software to allow access to binary data files from user written

22 C programs. It s a good idea to set up a short-cut on the desktop from which to start the Eye Tracker. It s also a good idea to keep track of which version of the Eye Tracker software you are using. Getting technical assistance may require having this information. The Gaze Tracker program installs itself on the Display PC in C:\ERT\bin\> The file is named GazeTracker.exe. 2.2 Interface Description The Eye Tracker Interface PC is connected to the Model 5000 Control Unit with a serial, RS232 cable from COM1 on the PC to the Controller connector on the model 5000 control unit (see Figures 2-2, 2-3, and 2-8). If an optional magnetic head tracker is being used it interfaces to the connector labeled Head Tracker, with a serial (RS232) cable (Figure 2-8). Real time serial data can be exported to an external device via connector labeled Serial Out (RS232). The interface protocol is described in Chapter 6. The video images from either the scene or pupil monitor outputs may also be videotaped. The XDAT port on the Control Unit may be used to input digital data from an external device, to indicate external events. These signals are numbered consecutively as they are received and inserted into the data stream. 2.3 Options Described below are some additional components and software options, available from ASL, which may further enhance system operation Binocular and High Speed 504 Eye position data is normally sampled at 60 Hz. The model 504 is available with a High Speed (HS) option, capable of 60, 120, and 240 Hz sampling rates. In addition there is a remote Binocular option consisting of two sets of optics to remotely track left and right eye movements Magnetic Head Tracking Hardware (MHT) The magnetic head tracking option (MHT) is a small unit that determines head position and orientation with good accuracy in six degrees of freedom (in X, Y, and Z dimensions). The head tracker output can be recorded independently of the eye tracker by a host computer or by the Interface PC. When used with the model 504 eye tracker system, the MHT is used by the Control Unit to assist in aiming the camera to keep the pupil in focus. The magnetic reference source is a stationary transmitter mounted behind the subject which generates a magnetic hemifield. Head position is measured in six degrees of freedom at distances of up to approximately 36 inches from the transmitter. The MHT option includes a control unit which powers the device, a magnetic transmitter (reference), and a small sensor or receiver which is attached to a Velcro head band worn by the subject. Nylon screws are provided for mounting the magnetic transmitter module. A cable is provided to connect the MHT control unit to the Head Tracker port on the Control Unit (see Figure 2.8). ASL will supply a tripod on which to mount the MHT transmitter and position it behind

23 the subject. However, if circumstances permit, it may be advisable to permanently mount the transmitter on a wooden shelf located behind the subject to prevent accidental movement of the transmitter. If the transmitter is moved, the system must be re-calibrated. MHT performance can be affected by large pieces of nearby metal or anything else, such as an AC power line, that causes magnetic field distortion. Consult ASL for details.

24 Chapter 3: Installation 3.1 Unpacking and Assembly Setting up the apparatus may be aided by consulting Figures 2-1, 2-2, and 2-3, in the previous chapter and Section 3.3 and Figures 3-11 through 3-13 in this chapter. Except for the connectors on the back of the Interface PC and the Display PC, which are supplied by the user, the connectors on the Control Unit, Scan Converter, MHT controller, the camera, and the two black and white monitors are marked with colored tabs. The cable-ends, including the RS232 serial cables which connect to COM 1 on the user-supplied PC s, are marked with matching colored tabs. Connecting the cables is described in Section Control Unit Locate the model 5000 Eye Tracker Control Unit and unwrap the protective material. Be sure that the power switch on the front panel is in the OFF (down) position (Figure 2-7). For use with model 504 pan-tilt optics, be sure that the slide switch on the rear panel is in the P/T position (Figure 2-8). Locate the 12VDC power supply module and connect it to the DC Power connector on the Control Unit. Connect the other side to an AC power outlet. The power supply is input rated for VDC, 50 or 60 Hz. The LED on the front panel of the Control Unit should remain off until the power switch is switched on Pan Tilt Optics The pan tilt module will be in a small box in one of the large shipping boxes. It is packed in protective foam along with a 13.5 VDC power supply, remote control, and the removable tilt knob. Place the optics module on a flat surface or, if it will be mounted to a tripod, fasten it to the tripod using the 1/4-20 hole under the nonmoving base. Be sure the power switch is in the off position. Connect the 13.5 VDC supply to the DC IN connector Scene Camera If a scene camera (FMSC) was ordered from ASL, locate the scene camera in one of the shipping boxes. It will be a very small camera about 1in. sq (2.54cm sq). Locate the scene camera tripod and adjustable tripod head. The tripod will usually be in the manufacturer s box in one of the two-foot cube boxes. Mount the scene camera on the tripod. The handle on the tripod head, which turns to loosen and tighten, may be used to position the camera Scan Converter If a scan converter (AverKey Micro) was ordered from ASL, find the Scan converter box, carefully unpack the components and place them near the Display Computer and monitor. Note that use of the scan converter is appropriate only if the eye tracker subjects will be looking at a VGA computer display. There is no mechanical assembly required; cable connections are described in Section 3.3 ( Interconnections ) Eye Tracker Interface PC

25 If the Eye Tracker Interface PC has been supplied by ASL, locate the computer, monitor, keyboard, and mouse, and assemble in the usual fashion. If supplied by ASL, the computer will have the eye tracker software pre-installed. Software installation is discussed in Section 3.4. The Eye Tracker program is discussed in Chapter 4. The instructions for installing the Eye Tracker software are also supplied with the program CD. It is assumed that the user is familiar with standard PC assembly and operation. If not, please consult ASL Magnetic Head Tracker The magnetic head tracker (MHT) unit comes individually packaged. The package contains a control box, a source module with cable and connector, a sensor module with cable and connector, and a manual with one or more floppy disks. The MHT system may be either an Ascension Flock of Birds or a Polhemus FASTRAK or ISOTRAK type system. Consult ASL for comparative details. The MHT transmitter (source) module must be mounted to a stable, nonmetallic surface using the nylon screws provided. The transmitter should be mounted so that it will be just above and behind the subject s head. When the subject s head is in the nominal center (or average) position, the sensor worn by the subject should be inches from the transmitter. Placement and mounting of the transmitter is discussed in Section and illustrated in Figures 3-9 and If possible, the transmitter should be oriented so the x-axis is directed toward the subject and so that the y-axis points down. The front of the transmitter, which faces toward the subject, has a raised, contoured face. It is also very important that the transmitter be rigidly supported, since any motion or dislocation of the transmitter will introduce errors into the data. It is a good idea to mark the sensor, worn by the subject, with an arrow showing the up (Y+) direction (Figure 3-1). During calibration, the software is taught left/right, up/down, and near/far directions. During data collection, MHT calibration information is used to track the subject within the three-dimensional field. Failure to position the sensor in the same orientation as it was held during calibration will result in errors in the data.

26 Figure 3-1. MHT Sensor marked to show up (Y+). 3.2 Component Placement Optics and Scene Relationships The standard placement of the pan tilt module and the resulting measurable field of view - the field within which accurate measurements may be made - is shown in Figure 3-2. The measurable field of view is shown in another way by figure 3-3. Figure 3-2. Measurable Field of View

27 Figure 3-3. Measurable Field of View from Subject s Point of View. The measurable field extends symmetrically to either side of the camera and above the camera centerline as much as 25 degrees, but only extends a small distance below the camera. This is because the upper eyelid begins to occlude the camera s view of the eye when the subject looks below the camera. For this reason, ideally the camera module should be placed near the bottom of, or just below, the desired field of view (Figure 3-4), near the horizontal center of the field. Such an arrangement minimizes eyelid occlusion of the pupil and provides left to right symmetry. Figure 3-4. Optimal Placement for Camera and Monitor.

28 If the line of sight is perpendicular to the surface being viewed, the visual angle, theta (2) between two points on a flat surface may be calculated as: θ = tan 1x d When theta is small (Figure 3-5), less than about 10 degrees, the calculation can be simplified as: θ = x 180 d π When the line of sight deviates from the axis perpendicular to the surface of the scene (bottom of Figure 3-5), theta is calculated as: θ = tan x+ x' tan d 1 1 x d ' Figure 3-5. Computation of Visual Angle between Points on a Flat Surface.

29 Less than ideal positioning may be required for a particular application or experiment. Figures 3-6, 3-7, and 3-8 show typical placements of the components. Figure 3-6. Camera placed beside Monitor. Figure 3-7. Camera placed in front of a very large Monitor.

30 Figure 3-8. Camera mounted on Tripod in front of Projection Screen. The pan-tilt camera may be placed anywhere so long as the following constraints are observed: 1. The optics unit to eye distance ( d" in Figure 3-2.) must be in the range of inches. The optimal distance is from 26 to 32 inches. There are trade-offs here. If the distance d is smaller, the eye image captured by the close-up lens fills the monitor screen and reflection of stray light from other areas of the subject s face is minimized. But under these conditions, if the subject moves during data collection, compensatory tracking motions of the camera have to go through a greater angular distance and require slightly more time to track the eye. 2. There must be a clear optical path between the optics unit and the eye. 3. Wherever the camera is placed, the measurable field of view will be about 25 degrees visual angle to either side of the optics, about 25 degrees above the optics and about 10 degrees below the optics as shown in Figures 3-2, and 3-3. The method for calculating visual angles using measured distances is shown in Figure If the subject is seated in front of a computer monitor (Figure 3-4), the person s eye should line up just above the middle of the monitor screen. A chair, such as an office or secretary s chair with a height adjustment, may facilitate accurately positioning the subject. 5. Although the distance between the pan-tilt camera and the eye is restricted, there is no restriction on eye-to-scene distance. The scene may be a video display, a slide projection, or any other display or natural scene. The scene may be any distance from the eye, so long as the area to be scanned falls within the visual field defined by Figures

31 3-2, 3-3, and 3-5. Errors in point of gaze measurement due to head motion will be minimized, however, when the scene being viewed by the subject is at the same distance from the subject as the front of the pan-tilt module. Such optimal placement may not always be practical, and other arrangements do not preclude successful gaze measurement. In Figures 3-4, 3-6, and 3-7, the scene being viewed is a monitor. Note that only figure 3-4 satisfies the optimal conditions by placing the optics module and scene surface the same distance from the subject, and placing the optics module just below the scene but centered horizontally. Figure 3-6 shows the optics placed to one side of the monitor. This will work well so long as the upper right corner of the monitor remains within about 25 degrees visual angle from the pan-tilt camera. If the surface to be viewed by the subject is especially large, like the very large monitor shown in Figure 3-7, the optics module may be placed in front of the monitor. If the pan-tilt module were placed below this large monitor, points at the top of the monitor screen would probably be more than 25 degrees visual angle from the pan-tilt optics, and gaze directed at the top of the monitor screen would be beyond the measurable field. In other cases, such as a subject viewing a large projection screen, the pan-tilt optics module may have to be even further from the scene surface, as shown in Figure 3-8. Regardless of how the camera and viewing surface are positioned, there must be a method for displaying a nine-point calibration target on the display scene surface. Display of the calibration target is explained in Section If a separate scene camera is used to record the scene being viewed by the subject (Figure 2-1), it should be placed as close to the subject s head as possible so that the scene camera image shows the same perspective as that seen by the subject. A scene camera is not required at all if the source of the scene is a video recording, presented using a scan converter (Section and Figure 2-2). Although a tripod is provided, a scene camera may be mounted in any way that is convenient. It is recommended that the scene camera be mounted securely enough so that its position will not be changed accidentally by someone bumping it or brushing against it. As described in Chapter 5, system calibration requires that the scene camera be stationary; some calibration procedures must be repeated if it moves Magnetic Head Tracker Placement If the system is equipped with a magnetic head tracker (MHT), the MHT transmitter (source) module must be mounted to a stable, nonmetallic surface using the nylon screws provided. It is suggested that the transmitter be mounted so that it will be just above and behind the subject s head, as shown in Figure 3-9, so that when the subject s head is in the nominal center, or average position, the transmitter is about 6 inches behind the head. The transmitter creates a three foot magnetic hemi-field, so these distances are not rigidly prescribed. The transmitter should be oriented so the x-axis is directed toward the subject and the y-

32 axis points down. It is very important that the transmitter be rigidly supported, since any motion of the transmitter will introduce errors into the data. Figure 3-9. Placement of Magnetic Transmitter. A common mounting arrangement for the transmitter is a wooden post with a heavy metal or wooden base, as shown in Figure Mounting the transmitter to a wooden shelf attached to a wall behind the subject makes an arrangement that is even less susceptible to accidental movement. Figure Magnetic Transmitter Mounted on Wooden Post.

33 3.3 Interconnections Figure 3-11 shows the connections for the model 504 Eye Tracker with an optional magnetic head tracker. As noted at the beginning of this chapter, all of the cable connections are color-coded. Also see Figures 2-1, 2-2, 2-4, and 2-8. Figure Connections for Model 504 Eye Tracker with Magnetic Head Tracker.

34 3.3.1 Pan-Tilt Optics Module Connections 1. Be sure that the system power and power to the pan-tilt module are OFF. 2. Connect the 13.5 VDC power supply to the pan-tilt module DC-IN connector, and to an AC wall outlet. 3. Find the camera-to-control unit cable (white/red dot). The cable has a 25-pin D- connector at one and, and multiple connectors at the others. The standard length is 25 ft. (7.6 meters), although other lengths are available. 4. Plug the 25-pin D-connector into the connector labeled Camera (white/red dot) on the Model 5000 Control Unit (Figure 2-8). If there is a small cable with a BNC connector splitting off from the 25-pin connector, it should be connected to the A/B switch on the Scene monitor. At the other end, the Camera cable splits into five ends: A. Connect the plug with the yellow end into the socket labeled VIDEO OUT (yellow dot) on the rear panel of the pan-tilt camera; B. Connect the yellow/blue end to the socket labeled VISCA IN with (yellow/blue dot) on the rear panel of the pan-tilt base; C. Connect the white/blue end to the other white/blue cable extending out from the neck of the pan-tilt camera; D. Connect the green blue end to the small power supply with the came color code; E. Plug the remaining socket into the connector on the rear of the moving head on the pan-tilt assembly (see Figure 2-3 and 2-4). There may be a wire coming out of the back of the camera, instead of a connector on the camera itself. Strain-relieve the camera cable at the camera end by taping it to the table or tripod leg, etc. Cable ties or other strain relief techniques may be used. Be sure that the illuminator cable (the lead that is connected to the moving part of the pan-tilt mechanism) has enough slack to accommodate full panning motion of the mechanism. If the point where the cable separates into multiple leads is close to the pan-tilt rear panel there will be sufficient slack Wide Angle Locating Camera Lens The wide angle lens is used as an aid in locating the subject s eye. Due to the small field of view associated with the close-up lens on the pan-tilt camera, it can be extremely difficult to acquire a close-up image of the eye when the subject is being calibrated (to find the subject s eye in the close-up lens). The wide angle lens on the pan-tilt camera (see Figure 2-6) provides a separate video signal that does not pass through the Control Unit. The wide angle lens has an axis which is parallel to the axis of the close-up lens so the two move and point in parallel. The Scene Monitor shows the view through the wide-angle lens when the A/B switch on the Scene Monitor is switched to the B position (Figure 2-2). The Eye Monitor shows the view through the close-up lens.

35 It helps to aim the camera if the glass face of the Scene Monitor (switched to the wideangle lens) is marked with a small dot, using a magic marker, to indicate where the close-up lens is pointing. Use the handheld control to aim the wide-angle lens at a stationary object such as the MHT transmitter. Position the image of the object so it is in the center of the Scene Monitor screen. Find the object in the close-up lens and center it in the Eye Monitor. It may require moving the camera slightly until a shadow crosses the Eye Monitor, and then focusing the close-up lens to bring the object into view. With the close-up lens focused on the object, use the magic marker to make a small dot on the glass screen of the Scene Monitor to mark the center of the image of the stationary object. The small dot can later be used to center the wide-angle lens on the subject s eye Obtaining an Eye Image When a subject is seated, position the A-B switch on the Scene Monitor to the B channel to show a wide angle view from the camera. Use the Pan-tilt remote control to aim the camera at the subject. The Scene Monitor will show an image of the subject s face. The Eye Monitor will show a very small, magnified portion of what the wide angle lens sees. The Eye Monitor image may or may not be in focus. Use the Pan-tilt remote to position the eye in the center of the Scene Monitor, using the small dot as a guide. If the illuminator beam is ON, it will appear as a bright disk on the subject s face. Try to position the illuminator beam over the eye. Once the subject s eye is centered in the Wide angle field of view it should also be in, or near, the field of view of the close-up lens. Turn auto-focus off, and use the zoom buttons on the remote control to bring the image of the subject s eye into focus in the Eye monitor. Switch auto-focus back on Eye Tracker Interface PC Connections Use the cable provided (RS232 cable with green dot) to connect the port labeled Controller on the Control Unit to the COM1 port on the Interface PC. If it is necessary to use a different COM port please consult ASL Black and White Monitors Connect a video cable from the Eye Monitor video input (orange dot), to the Control Unit connector labeled Eye Out (orange dot). Connect a video cable from the Scene Monitor video input, (blue dot) to the Control Unit connector labeled Scene Out (blue dot) Magnetic Head Tracker Connection The MHT transmitter (source) and sensor modules attach to connectors on the MHT electronics unit. Connect one end of the MHT interface cable (yellow/red dot) to the RS232 port on the MHT control unit marked with a yellow/red dot. Connect the other end of the MHT interface cable to the model 5000 Eye Tracker Control Unit connector labeled Head Tracker, also marked with a yellow/red dot. Set the DIP switches on the MHT electronics unit for Baud, RS232 communications. All switches will be down except switch number 0ne which is up. (consult manufacturer s manual packaged with the MHT system for proper DIP switch

36 settings). The MHT system type (Flock of Birds) must be set in the interface program (e5win.exe) before trying to establish communications between the Eye Tracker Control Unit and the MHT. The system type can be set through the MHT pull down menu in the Eye Tracker program (see section 4.2.4) Direct Use of Scene Video If the subject will be viewing a video image (e.g., a videotape displayed on a monitor) the same video signal can be used directly as the scene video signal for the eye tracker, rather than using a separate scene camera. If the subject will be viewing a computer screen, a scan converter can be used to convert the computer image (e.g., VGA) into a composite video signal, which in turn can be used as the eye tracker scene video signal. ASL can supply a scan converter instead of, or in addition to a scene camera. Connections for these configurations are shown in figures 3-12 and 3-13 (also see figure 2-2)

37 . Figure Connections using Scan Converter to supply Scene Video Signal.

38 Figure Connections using Scene Video Signal in place of Scene Camera. 3.4 Software Installation The Eye Tracker software is shipped on a CD. The software will be installed on the hard drive if the Eye Tracker Interface PC has been supplied by ASL. If the computer was not supplied by ASL, locate the CD labeled Eye Tracker Software and follow the directions on the label to install. The disk should auto-run when it is put into the drive. As discussed in Section 2.1.5, system requirements for the Interface Computer are an IBM compatible PC capable of operation with Windows 95, Windows 98, NT, 2000, or XP if using the Windows eye tracker interface, or DOS if using the DOS eye tracker interface. The computer must have available COM1 or COM2 serial ports using standard interrupts and device addresses. The minimum recommended system is a 200 MHz Pentium, but slower systems will

39 work, so long as they can effectively run Windows 95, 98, or NT. Running other simultaneous applications that take up a significant portion of the PC s processor time may cause eye data to be lost. When not recording data the only consequence of the PC not having enough time will be a sluggish interface program display. Operation of the Eye Tracker program is discussed in the next chapter.

40 Chapter 4: The Eye-Tracker Interface Software All of the original Eye Tracker manuals are available as.pdf files on the hard drive on the Interface Computer, in C:\Program Files\ ASL Eye Tracker\ Manuals. The Eye Tracker software package contains the Eye Tracker user interface program (E5win.exe) that runs on the Interface PC. The program contains configuration information that must be uploaded to the model 5000 Control Unit to establish communication among the system components. The following sections assume that all appropriate interconnections have been made and software installed, as described in Chapter 3. This manual also assumes that the Windows interface program (e5win) will be used. ASL can provide a manual upon request that describes the operation of the DOS Interface program (e5000). 4.1 Uploading Program Information to the Control Unit Power up the model 5000 Control Unit. If it is already powered up from a previous use, use the power switch to cycle it off and back on again to be sure that it is reset and any stored information has been erased. Open the Eye Tracker Interface program (e5win) by: clicking on the Start button /Programs /ASL Eye Tracking /Eye Tracker Interface; double clicking a corresponding desktop shortcut; or; using Windows Explorer to navigate to the Eye Tracking directory (under C:\ProgramFiles\ASL Eye Tracker\EyeTracking\>) and double clicking e5win.exe. The Eye Tracker interface program main screen will appear along with a pop up window called Upload to Series 5000 Control Unit (Figure 4-1). If necessary, use the COM Port menu box to specify the PC COM port actually being used (the default is COM1). If necessary, use the Baud Rate menu box to set the proper baud rate. These settings are stored with the program. Once they are set up, they will carry over each time the program is started. Note: ASL Series 5000 Control Units with serial # 1264 and higher have a baud rate of 115,200 K baud. ASL Series 5000 Control Units with serial # 1263 and lower have a baud rate of 57,600 K baud. The FPGA and DSP files contain information that is written to the Control Unit once Upload is begun. The file names should automatically be set to the proper files. These files may be updated when a new version of Eye Tracker is installed. Click Start Upload. Progress bars on the Upload window will show programming information being uploaded, first to the field programmable gate array (FPGA), and then to the digital signal processor (DSP) in the Control Unit. After a successful upload, the pop up window will disappear and the green On Line light at the upper left of the screen will be on. If errors are encountered, the pop up will not close, and the scrolling upload display window will contain an error message with a suggestion for how to proceed.

41 Once the Control Unit has been loaded, the information it contains will continue to run until the power is turned off. It is not possible to reload software to the control unit until the unit has been powered off and back on to reset it. Recycle the Control Unit by turning the power off, wait for 15 or 20 seconds, then turn the unit back on. The Upload process may be re-started by clicking the Upload button on the interface, or by selecting upload to control unit from the Eye Tracker Configuration menu. If an attempt is made to reload the control unit without first resetting it, a pop up message will prompt the user to re-cycle power to the Control Unit. The user interface program (e5win) is used to change settings, such as pupil and corneal reflection discrimination thresholds, to launch procedures such as calibration, and to record data on the Interface PC. All eye tracking functions are performed by the model 5000 Control Unit. Once proper settings are established, and if data is not being recorded, the eye tracker will continue to function normally even if the cable to the Interface PC is disconnected. 4.2 E5000 User Interface Program Figure 4-1. Eye Tracker Main Screen with Upload Dialog Box. The E5000.exe program is a DOS version of Eye Tracker that can be used in place of e5win.exe, the Windows version. Simply double click e5000.exe, in the EYEPOS directory, instead of e5win.exe. A separate manual is available from ASL that describes system operation in terms of the DOS Interface. Here it is assumed that e5win (windows version) is

42 being used User Interface Screen Once the FPGA and DSP information has been successfully uploaded to the Control Unit, the green Online light, near the top left of the interface program screen, should be on. If this light remains red, the Upload was unsuccessful and must be repeated. The main interface program window has a menu bar at the top, a shortcut bar directly underneath the menu bar, a column of controls extending down the left side of the screen, two graphics windows labeled Eye and Scene POG, and some additional digital information displayed below the graphics windows (Figure 4-2). A list of the menu choices appearing under each pull-down menu - File, Configure, Calibrate, etc. - is given in Appendix A2. The controls on the left side of the screen consist of standard windows slide switches, Figure 4-2. Eye Tracker Main Screen - Upload Complete. check boxes, and radio buttons. The data displays consist of indicator lights to indicate online or off-line, valid pupil and CR recognition (both red in Figure 4-2), and text windows displaying COM port assignment, PC time of day, elapsed time, free disk space, point of gaze coordinates ( Scene POG ), pupil diameter, external data values (XDAT), and data file information. The on-line indicator light and COM port assignment displays are at the top of the control column. Other data displays are beneath the two graphics windows. When the

43 magnetic head tracker is connected and enabled, an additional data display opens beneath the Scene POG window, showing the magnetic sensor position and orientation coordinates in X, Y, and Z dimensions. When the system is operating correctly and after a subject has been calibrated, the Eye graphics display box at the top center of the screen shows pupil and corneal reflection (CR) center positions and diameters as detected by the eye tracker. The pink pupil center cross hairs and blue CR center cross hairs are essentially the same as the white and black cross hairs that show pupil and CR centers on the Eye Monitor. The pupil and CR circles shown on the screen are not the same as the pupil and CR outlines displayed on the Eye Monitor. The computer display simply draws circles about the cross hair positions with diameters proportional to detected pupil and CR diameters. These circles do not show the actual pupil and CR outlines detected by the eye tracker. True outlines are displayed on the black and white Eye Monitor. A after a subject has been successfully calibrated, the graphics window labeled Scene POG shows a point of gaze cursor. This is essentially the same as the cursor movement on the black and white Scene Monitor. Target points are also displayed in the Scene POG window. During data collection, the Scene POG window will show active eye movements, but these are not superimposed on the scene as viewed by the subject. The black and white Scene Monitor, switched to A, shows the monitor screen or other scene the subject is viewing, with eye position cross hairs superimposed. If the Model 5000 Control Unit is not running (powered off or reset) or if the cable connecting the Interface PC to the Control Unit is disconnected, the light labeled On line, at the top left of the screen will turn red. If the communication is reestablished, for example by reconnecting the cable, this light will change back to green System Settings Pull down the Configure menu by clicking on Configure in the menu bar at the top of the screen, and select System Settings to pop up the System Settings dialog window (Figure 4-3). Use the radio buttons under System Type to select the system type that reflects the current hardware configuration (Pan-tilt optics with MHT option). Set the eye camera speed (60 Hz) to correspond to the update rate of the eye camera being used by clicking on the down arrow next to the corresponding camera speed box and selecting from the drop down list. (If an optional high-speed camera is being used, the camera dip switches must be properly set for the selected update rate). The Eye Camera Speed setting in the System Settings window does not control the camera update rate, it merely informs the program of the type of eye camera being used.

44 Figure 4-3. System Settings Dialog Window. Using the drop down list in the System Settings window, assign a COM port (usually COM1) as the interface program port. Be sure that this assignment corresponds to the physical connection between the Com port on the Interface PC and the Controller port on the eye tracker Control Unit. Before being saved or output as valid X-Y eye position data, the individual samples are averaged. The number of samples being averaged is specified by the item labeled Number of eye position fields to average (set to zero in Figure 4-3). Simply type in the desired number of samples to be averaged. The recommended value for the pan-tilt optics system is 4. This means every eye position value output to the data file will be averaged with the previous 3 values before being displayed or recorded. To eliminate any averaging, enter 1 or 0. Only

45 gaze coordinates are averaged; neither pupil diameter nor any other recorded values are averaged. Sample averaging is not the same as the criteria used to determine the start and end of an eye fixation, and its location. These criteria are discussed in Chapter 7 for Eye Tracker and Chapter 14, Section for Gaze Tracker. It is important to note that after a period during which a pupil was not recognized (no valid gaze measurement could be made) the first valid measurement is not averaged. The next measurement is averaged with the previous valid field, and the number of fields averaged increases in this way until the specified value is reached. The check boxes on the System Settings window usually should be left unchecked. The 17-point calibration capability is explained in section and the auto Eyedat function is explained in Chapter 7. Use Metric System applies only to setup of a magnetic head tracker for use with pan-tilt optics, as discussed in section The other two check boxes do not apply to pan-tilt optics and should be left unchecked. The Scene Video Source radio button should normally be left on Auto Select when using pan-tilt optics. When System Type is set to Pan-tilt Optics or Pan-tilt Optics with MHT option the system will automatically select the Remote Scene connector (on the Control Unit rear panel) as the scene video source. When using head-mounted optics with a head mounted scene camera, the scene video signal is usually part of the cable that connects to the Camera connector on the Control Unit. Click OK to save the system settings and close the System Settings dialog window. If program information has been properly uploaded to the Control Unit and the COM ports have been properly assigned, the Online light near the top left of the main screen should be green to indicate that the PC is communicating with the Control Unit. If this light is red, indicating lack of communication between the PC and the Control Unit, check connections and try uploading software to the Control Unit again. (To upload, turn off the Control Unit, exit the interface program and follow the directions in Section 4.1) Saving and Retrieving Default Data The current subject calibration data and system configuration settings are stored in the same directory as the E5Win program, in files called E5000.CAL and E5000.CFG, respectively. When a pan-tilt optics module is being used (model 504), information about the pan-tilt module settings and calibration information - the camera s location with respect to the magnetic head tracking system (MHT) - is stored in a file called E5000.PTC. (Calibration of the MHT is explained in Chapter 5.) These files are automatically updated whenever changes are made to the data they contain, and are loaded whenever the Eye Tracker program is started. For example, E5000.CAL is automatically updated at the completion of every subject calibration. Thus, the last calibration performed is remembered and reloaded the next time the Eye Tracker program is run. Similarly, the E5000.CFG file is updated whenever <OK> is clicked to exit from the System Settings dialog window. To save one of these default files for future use, or to retrieve an old file previously saved, use the Save As or Read selections from the corresponding pull down menu. Access

46 the E5000.CAL file from the Calibrate menu; access the E5000.CFG file from the Configuration menu; and access the E5000.PTC file from the Pan-tilt menu. For example, to save a particular set of subject calibration data, use the Save As selection on the Calibrate menu. In response to the prompt, enter a file name other than E5000.CAL, such as PETER.CAL, and click <Save>. To use subject calibration data previously saved, select Read from the Calibrate menu, browse to the previously saved file, highlight it and click <Open>. It will be used as the current data until overwritten by a new calibration. The same file manipulations can be done with Windows Explorer. Simply rename the E5000.xxx file to prevent its being overwritten, and rename it back to E5000.xxx (in the same directory as the E5Win program) to re-use it Enabling the Magnetic Head Tracker The following section applies only if an optional magnetic head tracking (MHT) system has been connected as described in Sections and The MHT pull down menu has the following choices, shown in Figure 4-4: Figure 4-4. MHT Pull-Down Menu The Set boresight command will be grayed (inactive) until the MHT system is enabled. Before attempting to enable the MHT system for the first time, choose Select MHT System and be sure the pop up window, labeled MHT Type shows the type of MHT hardware actually being used (Figure 4-5). If not, activate the drop down menu and select the proper type. If unsure of the proper type, consult ASL. We are using the Ascension Flock of Birds in our system.

47 Figure 4-5. Selecting MHT Type. Click on Enable from the MHT pull-down menu to start communication between the Eye Tracker program and the MHT system. Alternatively, click the MHT button at the far right side of the shortcut bar. If communication is successfully established with the MHT system, the MHT data display, labeled Transmitter Offset will appear under the Scene POG display window and the MHT button will appear to be depressed (Figure 4-6). Figure 4-6. Interface Program Main Screen showing MHT Transmitter Offset.

48 The Enable menu selection as well as the MHT short cut button are toggle switches. Once the MHT system is enabled, the top item on the MHT menu changes to Disable and the MHT shortcut button appears activated (pressed down). Selecting Disable or clicking the MHT button will disable MHT communication with the Eye Tracker, and the MHT Transmitter Offset data display under the Scene POG window will disappear. The MHT data display consists of 3 position and 3 orientation values. The position values are the position of the magnetic sensor with respect to the transmitter x, y, and z axes. The orientation values are the azimuth ( az ), elevation ( el ), and roll ( rl ) angles (often called Euler angles) that describe the orientation of the sensor axes with respect to the transmitter axes. Position values are expressed in inches, and orientation values are expressed in degrees. If the MHT system is communicating properly with the Eye Tracker, the MHT display values should change when the sensor moves, and should match the actual position of the sensor with respect to the transmitter. Actually, the values will probably be constantly changing, even when the sensor is stationary, due to noise in the system. The Set Boresight command under the MHT menu causes the sensor coordinate frame to rotate so that sensor orientation angles are zero for the current sensor orientation. When MHT mirror tracking (as is the case with the model 504), boresights are done automatically at appropriate times, so this command need never be used during normal system operation. It may sometimes be useful, however, for checking to see that the MHT system is functioning properly. Some additional explanation of boresight may be helpful. The origin of the sensor coordinate frame is in the center of the sensor. Upon power up, or after a reset, the sensor coordinate frame x axis extends away from the sensor cable, and the y-axis extends down from the sensor mounting surface. If looking in the positive x direction with the y-axis pointing down, the z-axis extends to the right (Figure 4-7). The MHT system reports the orientation of the sensor coordinate frame relative to the transmitter coordinate frame. In other words, if the sensor is held so that sensor axes are aligned with transmitter axes, the MHT system will report zero orientation angles.

49 Figure 4-7. MHT Sensor Orientation. If the sensor is held still in any orientation and the Set Boresight command is issued, the sensor axes will be re-calibrated in the software to align with the transmitter axes. The sensor coordinate frame will maintain this new orientation until a reset is issued or until the unit is power cycled. The third choice on the MHT pull-down menu, Reset MHT, sends the same MHT initialization command string that is automatically sent during the MHT Enable operation. The effect of any previous boresight command will be canceled.

50 Chapter 5: Eye Tracker Calibration The following steps are necessary to successfully operate the eye tracker in the standard fashion: 1. The model 5000 eye tracker Control Unit must be powered up, the E5Win user interface (Eye Tracker) program must be started, and information must be successfully uploaded to the Control Unit. If an optional magnetic head tracker is being used it must be enabled (Chapter 4); 2. If appropriate values are not already stored in the default calibration file, a calibration target point pattern must be entered using the Eye Tracker program. Since computer screens differ, this calibration step indicates to the system where each point will lie on the screen. It need not be repeated each time unless the subject s display monitor has been changed; 3. The pan-tilt camera must be calibrated relative to the MHT transmitter unit. This is another calibration step that need not be repeated unless system components have been moved; 4. The pan-tilt camera must be properly aimed at the subject. Zoom and focus must be adjusted to clearly focus the subject s eye image in the Eye Monitor (Chapter 6); 5. Stable discrimination of the pupil and CR must be achieved using the slider controls in the Eye Tracker program (Chapter 6) or other adjustments as required; 6. A subject calibration procedure must be executed. This includes locking in the sensor-to-eye offset in the MHT system and calibrating the subject s eye movements using the nine-point calibration screen (Chapter 6); 7. The subject is presented with the display materials and eye movements may be measured (Chapter 7). The following sections describe calibration of the target points and MHT calibration. 5.1 Setting Up Calibration Target Points Overview During subject calibration, the subject looks at nine target points that are at known positions. The Set Target Points routine from the Calibrate pull-down menu is used to inform the Eye Tracker program of the screen location of the target points that will be used for subject calibration (Chapter 6). The location of these points must be re-calibrated if the subject monitor is changed. If a scene camera or other composite video source is being used, the actual distribution of the nine points are taken from the scene monitor image and entered into memory with the Set Target Points function. If the stimulus scene is a video monitor, the target pattern can be displayed on the same monitor. If the display is a projected slide, the calibration target can also be on a slide. If the scene is an instrument panel, the calibration target pattern can be created by positioning small pieces of tape in appropriate places on the panel. Other methods are possible, so long as the

51 system is informed of the location of the points that are to be used during calibration of the subject. If a scene camera is being used, the scene camera must be properly positioned to view the scene; this can be judged by observing the nine calibration target points on the scene monitor. The positional relationship between the scene camera and the scene must be constant for the calibration to remain valid. The scene camera should be stable and fixed. Generally, the target points cover about 80 percent of the scene monitor screen area and are separated by degrees visual angle horizontally, and degrees vertically. Optimally, the middle column of vertical and the middle row of horizontal points should be colinear and perpendicular. These are ideal specifications. Compromises will often have to be made with no serious consequences. All points must be numbered from left to right; 1-3 for the top row, 4-6 for the middle row and 7-9 for the bottom row (Figure 5-1). NOTE: If the scene monitor was supplied by ASL it may be modified to have a reverse/normal switch on its rear panel for potential use with head mounted optics. If this is the case, confirm that the scene monitor is switched to the normal video mode. If the scene monitor is not set for normal video display, the calibration target point numbering described in this section will be incorrect Set Calibration Target Points Figure 5-1. Nine Point Calibration Target. 1. Select Set Target Points from the Calibrate pull down menu, or click on the Set target point shortcut button near the right side of the shortcut bar. The Set Target Points icon is a nine-dot pattern on a gray field (Figure 5-2).

52 Figure 5-1. Set Target Points Shortcut Button. 2. The Set Target Points window will pop up (Figure 5-3). Use the mouse to drag the pop-up window to a convenient position on the screen. Do not let the Set Target Points window cover any portion of the Scene POG window on the Interface program. 3. Once the Set Target Points window is open, whenever the mouse cursor enters the Figure 5-3. Set Target Points Window. Scene POG window, it changes to a cross, and the cursor coordinates are shown as Scene X: nnn Y: nnn in the Set Target Points Window (Figure 5-4). In addition, when the mouse is moved within the Scene POG window, a cursor on the Scene Monitor is displayed in the corresponding position.

53 Figure 5-4. Setting Target Points. 4. Display the nine-point calibration display and locate each of the nine points. There is a nine-point calibration slide in C:\Photos\ASLsampleCal.bmp on the C: drive in the Display Computer. The Scene Monitor displays the scene image (but the Scene POG window does not), whether it is a digital image captured through a scan converter or from a scene camera. The nine-point calibration display should be displayed on the video Scene Monitor during this procedure. Move the mouse within the Scene POG window so the cursor on the Scene Monitor is over point 1. Left click to enter point 1. Similarly, left click on points 2-9 to enter the other points. If it is hard to left-click the mouse without moving the cursor, use the Enter key or the Space bar instead. After moving the cursor to the correct point, use the up or down arrow keys to highlight the Store Position of Current Point button on the Set Target Points window, and use the <Enter> key instead of the left mouse button to enter the position of the point. The point shown in the Specify Position for: box should increment automatically as each point is entered. Be sure the correct point is indicated in the Specify position for: box before leftclicking or pressing Enter. NOTE: The Scene POG window displays nine default points left over from the last calibration. The position of these points will be overwritten as new target point positions are entered. If the default points remain visible after entering points 1-9, repaint the desktop by selecting the Misc pull down menu and choose Repaint Desktop.

54 5. If a scene video image is not available, use the position of the cursor in the Scene POG window, or the coordinate values on the Set Target Point window, to determine the target point positions. To enter target points out of order, click the small up or down arrow buttons at the right of the Specify position for: indicator to set the desired point. 6. Once all nine target points have been entered, click Save and Check Target Points to save the points and bring up the Check Target Points window (see next section), or Save Target Points and Quit to save the points and close the Set Target Points window Check Calibration Target Points Pop up the Check Target Points dialog window by selecting Check Target Points from the Calibrate menu, or by clicking the Check Target Points icon on the shortcut bar (Figure 5-5). Figure 5-5. Check Target Points Button. Use the up and down arrows next to the Current Point: indicator to select the desired point (Figure 5-6). The indicated point will be displayed in the Scene POG window and the Scene Monitor. If the cursor does not position itself over the target point in the Scene Monitor, the point needs to be re-calibrated. Go back to Section 5.1.2, open the Set Target Points window, use the up/down arrows to select the target point that needs to be re-set, set the point correctly, and click Store Position of Current Point. Only the points that are off need to be recalibrated. When all the points are correctly calibrated and checked, click on Set Target Points in the Check Target Points window to exit.

55 Figure 5-6. Check Target Points Window. 5.2 Calibration of Magnetic Head Tracker The magnetic head tracker (MHT) is used to assist pan-tilt tracking. Normally, rotation of the pan-tilt camera is driven in response to motion of the eye image in the Eye Monitor. When the image moves toward one side of the eye camera s field of view, the camera rotates in an appropriate direction to re-center the image. However, optical tracking will not work once the eye image is lost from the eye camera s field of view, for example, if the subject turns away from the camera. A system equipped with the MHT option uses information about the subject s head position within the magnetic field generated by the MHT transmitter to find the eye and recapture the image, even when the image has been lost momentarily by the eye camera Getting Started For the MHT-assisted tracking to function properly, the computer must know where the pan-tilt optics are with respect to the magnetic transmitter (source). Furthermore, it must know the relationship between MHT coordinate space (defined by the orientation of the magnetic transmitter s X, Y, and Z axes), and pan-tilt coordinate space (defined by the mechanism s pan and tilt angles). The calibration procedure provides the Eye Tracker program with this information. The MHT calibration procedure need be performed only once for a given magnetic transmitter and pan-tilt module placement. If either the camera or the transmitter are moved, the calibration must be redone. The MHT calibration is divided into two procedures: MHT Transmitter/Pan Tilt calibration and Sensor to Pan Tilt Calibration. The following description assumes that the transmitter has been mounted and that the MHT electronics unit, along with all other components, have been connected as described in Section Switch on the power switch at the rear of the pan-tilt module. The camera will turn all the way in one direction and will then return to a center position. The unit powers up in the auto focus mode. Start the Eye Tracker program and Enable the MHT system, as described in Section Be sure the Illuminator check box in the Eye Tracker program is checked to turn the illuminator on (Figure 4-2 and 4-6), and move the Illumination slider to at least midway. Move the Pupil and CR Discrimination slide switches all the way to the left. Be sure that the Pan-tilt Tracking button (near the bottom left of the Interface screen) is set to Manual and that the System Type button in the System Settings window (Figure 4-3) is set

56 to Pan-tilt optics with MHT option. The MHT calibration procedure will probably require two people, one person to work the keyboard and another to hold the magnetic sensor in various positions MHT Transmitter to Pan-Tilt Calibration Aim the Pan-tilt camera at the center of the Transmitter. Pull down the Pan-tilt menu and select Setup to open the Pan-Tilt Setup box (Figure 5-7). The Zoom, Gain, and Iris slide switches should be all the way to Max, and the Shutter slide switch should be at Long, as shown in the Figure. The slide switches can be moved by dragging them with the mouse, by clicking the right or left arrows on either end of the slide, or by clicking in the slide channel on either side of the current slide switch position. As noted previously, the red arrows can be used to move the camera from within the software. The operation is slow, however, and use of the hand-held control is recommended instead. Figure 5-7. Pan-Tilt Setup Window. Switch the Scene Monitor to the wide angle locating lens (Section 3.3.2) and use the remote control to aim the pan tilt camera at the magnetic transmitter. Imagine a line from the pan-tilt lens to the center of the transmitter. This imaginary line would intersect the front face of the transmitter a bit below the center of that surface (Figure 3-9). It is approximately this point on the front surface of the transmitter that should be centered in the eye monitor. Switch to Manual Focus and focus the closeup lens on the transmitter. Alternatively,

57 leave the camera on Auto-Focus and hold a finger, pencil, or some other object in front of the camera about inches from the camera lens. The system should be able to focus on the object at this distance. Slowly move the object away from the camera towards the magnetic transmitter. Once the object gets to more than about 22 inches from the camera, the camera may not be able to focus well. Move the Zoom: slide to the left until focus is restored. The zoom slide will not need to be moved far; it is probably best to move it by clicking the left arrow, at the left side of the slide, rather than by dragging the slider with the mouse. In this fashion try to achieve focus when the object is at about the same distance from the camera as the magnetic transmitter. Pull down the Pan-tilt menu and select MHT Pan-tilt calibration. (Note that this selection will be grayed out if the MHT system is not enabled.) A pop-up dialog box will appear asking MHT Transmitter/Pan Tilt Calibration? (Figure 5-8). Click Yes. Enter the distance, in inches, from the pan tilt camera to the MHT transmitter. Figure 5-8. Confirm MHT Calibration. Use a tape measure to measure the distance from the optics module center of rotation to the center of the magnetic transmitter. The camera s center of rotation is about 2 inches (5.1 cm) behind the plate that holds the illuminator LEDs (Figure 5-9). A measurement can be made from this front plate, next to the lens opening, and then 2.5 inches can be added.

58 Figure 5-9. Measuring Distance from Center of Transmitter to Pan-Tilt Camera Center of Rotation. The desired value is the distance to the center of the transmitter, but of course the tape measure can only be held against an outer face of the transmitter. Estimate the distance to the center as closely as possible by holding the tape measure along the side of the transmitter, or by separately measuring to the front surface of the transmitter and then, along the side of the transmitter from the front surface to the approximate center. Enter the appropriate value in the dialog box (Figure 5-10). The value must be entered in units of inches, unless the Use metric system check box has been selected in the System settings dialog window (Figure 4-3 and Section 4.2.2). If the value being displayed in the dialog box is correct, no change is necessary. Click OK. Note: selecting Cancel leaves the currently stored value unchanged. Figure Set Camera to Transmitter Distance. Place the sensor between the pan-tilt camera and the transmitter. Orient the sensor in the same way it will be oriented when worn by the subject (Figure 3-1). Hold the sensor in the same orientation at each station throughout the calibration procedure. A new dialog box will appear with the message Place sensor between pan tilt and transmitter (Figure 5-11). With the camera still focused on the transmitter, hold the sensor about 18 inches from the transmitter and slowly move it - without moving the camera - until it appears in the center of the Eye Monitor. With the sensor centered in the eye monitor, click OK.

59 Figure Place Sensor Between Camera and Transmitter. Since the pan-tilt optics are aimed directly at the transmitter, centering the sensor in the Eye Monitor places it along the line (vector) connecting the camera lens with the transmitter. This tells the system the direction of the vector connecting the transmitter and the camera. A pop up box will then announce MHT Transmitter -to-pan-tilt Cal complete. Click OK and proceed to the next section MHT Sensor to Pan-Tilt Calibration Auto-focus should be ON during this procedure. Completing this procedure allows the system to compute the relationship between the pan-tilt camera and the MHT coordinate frames. It involves holding the sensor so it appears in the middle of the pan-tilt camera frame at several different positions. By this means, the system is informed as to Left/Right, Up/Down, and Near/Far relative to the vector connecting the center of the camera with the center of the transmitter. It is important that the sensor be centered in the Eye Monitor as accurately as possible during this procedure. A dialog box will appear asking MHT Sensor/Pan Tilt Calibration? (Figure 5-12). To continue with the MHT calibration procedure click Yes. (To abort the procedure, select No ). Figure Continue MHT Calibration.

60 If Yes is selected, a new pop up window will prompt, Place sensor LEFT (Figure 5-13) and the camera will rotate horizontally to the left. Figure Place Sensor Left. Move the sensor to the left so it comes into view in the center of the Eye Monitor. With the sensor centered in the Eye Monitor, click OK. The command window will now prompt Place sensor RIGHT and the camera will turn horizontally to the other side of the centerline. Move the sensor along this horizontal line until the sensor is about 6 inches to the right of the center position. When the sensor is centered in the Eye Monitor display, click OK. The command window will next prompt Place sensor UP and the camera will point upward from the centerline. Move the sensor about 6 inches above the nominal center position until it is centered in the Eye Monitor and click OK. The command window will prompt Place sensor DOWN and the camera will point below the centerline. Move the sensor about 6 inches below center until it is centered in the Eye Monitor and click OK. During each of these movements, the sensor is held at about the same distance from the camera as at the start, and is oriented in X-Y-Z dimensions the same way at each station. The camera will move back to the center position and the command window will prompt Place sensor CLOSE. Move the sensor to the center position so it is centered in the Eye Monitor and move it several inches closer to the pan-tilt optics module, but not so close that auto-focus can no longer focus on the sensor. Do not reposition the pan-tilt, just move the sensor so that it remains centered in the eye monitor. Click OK. The command window will prompt Place sensor FAR. Without moving the pan-tilt module, move the sensor back to center position, and then several inches further away from the pan-tilt optics module, but not so far that auto focus can no long focus on the sensor. Move the sensor so that it remains centered in the eye monitor. Click OK.

61 After the FAR point is entered, the system will automatically do the necessary computations. If the computations are successfully completed, a pop up box will display MHT Sensor/Pan Tilt Calibration complete. Click OK. The above procedure can be aborted at any point by clicking Cancel. Points can be entered out of order by clicking the small up down arrows next to the box that specifies where to Place sensor Checking the calibration procedure. Be sure that the System type radio button on the System settings dialog window (Figure 4-3) under the Configure menu is set to Pan-tilt optics with MHT option. Use the Pan-Tilt Setup menu (Fig. 5-7) to turn Field Tracking Limits OFF (box labeled Use MHT Head Rotation Limits not checked). Field Tracking Limits are used during data recording. When the MHT limits are on, the program will not try to record data if the MHT moves outside of the limits. This prevents recording of spurious data. Turn the pupil and CR discriminators on the main interface screen OFF (slide switches on the main screen all the way to the left). Switch pan-tilt tracking to manual (check box at lower left of the main interface screen). Hold the sensor about where a subject s eye will normally be. Use the pan-tilt remote control to center the sensor in the eye monitor. With the sensor centered in the Eye Monitor, pull down the Pan-tilt menu and select MHT Sensor Calibration, or click the short cut button. The icon is a drawing of threedimensional coordinate axes near the center of the shortcut bar (Figure 5-14). Figure MHT Sensor Calibration Button. A pop up box will display MHT Sensor Calibration Complete. Click OK. Note that the sensor should be centered in the eye monitor when the shortcut button or the menu item is clicked, not when OK is clicked in response to the Calibration Complete message. Set pan-tilt tracking to Auto and move the sensor to different positions within approximately the same field used during the MHT calibration procedure. The camera should automatically re-aim itself at the sensor each time the sensor is moved. Don t expect the system to exactly center the sensor image in the eye monitor when doing this test. If it brings the sensor image into the eye camera field of view, even at the very edge of the monitor, the calibration can be considered successful.

62 If the calibration is not successful and the camera does not track correctly, repeat the steps in Section 5.2.3, Sensor to Pan-Tilt Calibration. If still not successful, repeat the procedure in Section 5.2.2, MHT Transmitter to Pan-Tilt Calibration, as well as the procedure in this section. MHT calibration results are stored in the E5000.ptc file. If a very good calibration is achieved for a particular set up, it can be copied to a different file name for safe keeping. If the E5000.ptc file is then accidentally changed or destroyed, the archived copy can be copied back as E5000.ptc.

63 Chapter 6: Subject Setup and Calibration Subject set-up consists of seating the subject in an appropriate position, obtaining a suitable eye image on the eye monitor, and setting the pupil and CR discriminators properly. Some trial and error may be required to develop a technique which will result in valid recordings for most subjects, but the experimentation should prove to be worthwhile. Although it will be impossible to record valid data from some subjects, a practiced experimenter should be able to successfully record four out of five subjects. 6.1 Subject Seating Any stationary chair can be used, but to the extent that the type of chair minimizes subject motion, eye-tracking accuracy will be maximized. Align the subject so a level gaze looks at about the center of the display screen. A chair with adjustable seat height, such as a secretary s chair, may help in making this adjustment. For testing the system, you can use the model eye target bar (a black plate with a white 4 mm spot and ball bearing). Place the model eye so the white spot is about where a subject s eye will be and focus the camera on the white spot. The model eye is useful for practicing set up and discrimination of the pupil and corneal reflection (CR) without requiring a cooperative subject. Use the velcro band to place the MHT sensor on the subject s head, with the sensor over the eye (usually the right) that will be tracked (Figure 3-1). It is possible to place the sensor anywhere on the subject s head that is deemed convenient, although usually over the forehead is the simplest. 6.2 Obtaining an Eye Image Check the Illuminator check box on the Eye Tracker Interface main screen to turn on the illuminator, and raise the illumination level by moving the slider labeled Illuminator to the right about half way. Be sure the Pan-tilt Tracking radio button, near the bottom left of the Interface screen, is set to manual and that the switch on the control unit rear panel is in the P/T position. Pull down the Pan-tilt menu and select Setup (Figure 5-6). The Zoom, Gain, and Iris slide switches should be all the way to Max and the Shutter slide switch should be at Long. These are the default positions for these slide switches. It is best to have them set to the default settings when first attempting to acquire a subject s eye image. Changes in these settings take effect immediately and remain set until they are changed. If the program is closed and re-started, the last settings, not the default settings, will be in effect Aiming the Camera With the Illumination on, use either the infra red remote control or the arrows on the Pan-tilt/ Setup window to aim the pan-tilt camera at the eye (or model eye). (See section on the use of the camera wide angle lens in aiming the camera.) The inner ring of arrows move the module a short distance for each click, while the outer arrows move the module a greater distance for each click. If the Reverse left/right and Reverse up/down boxes near the bottom right of the Pan-tilt/ Setup window are not checked, clicking an arrow moves the eye monitor

64 image in the direction of the arrow. If the Reverse boxes are checked then the pan-tilt camera will move in the direction of the arrow and the image (in the eye monitor) will appear to move in the opposite direction (as thought the window through which the image is being viewed were moving in the direction of the arrow). Camera movements may also be reversed using the hand-held control. The user should experiment to find out which setting of the reverse camera directions will be the least confusing to use during calibration of the subject. Once the subject s face is centered in the wide angle view, use the hand-held control to make small movements of the camera until the subject s eye is visible in the Eye Monitor. The image of the eye may be blurry at this point. As the eye comes into view in the Eye Monitor, tapping the arrow keys may be necessary to center the image of the eye Focusing the Camera The camera auto-focus is on when the camera is first powered up. Switch the autofocus to off using the hand-held control. There are two rocker switches on the hand-held control, one for fast movement of the field of focus, the other for slow movement (Figure 6-1). Pressing the top of the switch moves the image closer to the camera (T - Telephoto). Pressing the bottom button moves the image away from the camera (W - Wide angle). Slow movements are much easier to use. Using the rocker switches, it should be possible to bring the image of the subject s eye clearly into focus in the Eye Monitor. If the camera is unable to focus, move the Zoom slide in the Pan-tilt/ Setup window to Figure 6-1. Controlling Camera Focal Length. the left, toward Min, until focus is restored. The zoom slide will not need to be moved far; it is probably best to move it by clicking the left arrow once or twice. You should now see a clearly-focused image of the subject s eye in the Eye Monitor. Once the pan-tilt module is aimed at the eye or (or the model eye), pull down the Pan-tilt menu and select Set Home. This will cause the program to memorize the current position of the pan-tilt module. Subsequently selecting Home from the Pan-tilt menu will cause the module to

65 return to this memorized position. The Home position can be helpful for reacquiring the subject s eye when the system is not configured with the MHT option. The home button on the Pan-tilt remote control does not correspond to the Home settings in the Eye Tracker Interface menu and is not a substitute for it. Users with the MHT option can skip this step. If the MHT option is not being used, skip to Section Enabling the MHT System and Setting Sensor to Eye Offset This procedure does not require that valid Pupil and CR recognition be established, but the eye image must be clearly focused and centered in the Eye Monitor. Once the image of the eye is in focus in the Eye Monitor, completing the calibration described here allows the system to compute a vector connecting the center of the MHT sensor with the center of the eye. This allows the system to calibrate for differences in the shape of a subject s head, the location of the eye relative to the forehead, and the sensor position with respect to the eye. This procedure must be done for each subject. It is assumed the System Settings dialog window, under the Configure menu, is set to Pan-tilt Optics with MHT option (Section and Figure 4-3), and the MHT Transmitter and the Sensor-to-Pan-Tilt calibration procedures have been completed (Section 5.2). The mounting surface of the sensor and the head band worn by the subject are equipped with mating Velcro. The sensor should be mounted just above the eye being monitored (Figure 3-1). Enable the magnetic head tracker, using the MHT pull-down menu (Section and Figure 4-4). Once the MHT system has been Enabled, the Sensor-to-Eye vector shortcut button turns black (Figure 6-2). With the subject s eye image centered and focused in the Eye Monitor, click on the Sensor-to-Eye vector shortcut button. If the MHT system has not been Enabled, this button will be greyed out and clicking will have no effect. The eye image should be centered in the Eye Monitor when the shortcut button is clicked, not when OK is clicked in response to the Sensor Calibration Complete message.

66 Figure 6-2. Set Sensor-to- Eye Vector. The Sensor-to Eye vector button represents the X, Y, and Z axes of the MHT system. Clicking on the button causes the Eye Tracker program to calculate a vector from the MHT sensor to the center of the subject s eye. Once the vector has been established, the program uses the location of the sensor to find the subject s eye, should it be moved out of the camera s field of view. The Sensor-to-Eye Offset can also be set using the Pan-Tilt/ Setup window Set button (Figure 5-6), but this is a much less convenient method. A pop-up box will display MHT Sensor Calibration Complete and Pan-Tilt Tracking at the bottom of the Eye Tracker main screen will automatically switch to Auto. Click OK. The camera should now be able to reacquire the subject s eye automatically following a large head movement. If the system does not have a magnetic head tracker, leave Pan-tilt tracking set to manual until proper pupil discrimination is achieved as described in section Pupil and CR Discrimination The first stage in recognition of the pupil and CR by the Eye Tracker is performed by edge detection logic. The software seeks out bright areas within the frame of the Eye Monitor. The IR beam focused on the subject s eye is reflected by the retina ( red eye as seen in some flash photographs). The area of the pupil should be the largest bright area in the image, as seen by the software. The corneal reflection (CR) is a small point of reflected light, caused by the curvature of the eye, that occurs at the point where the cornea joins the white of the eye. This point is normally the second-largest bright point in the eye image. The program attempts to discriminate these light peaks from the surrounding illumination (Figure 6-3). If the thresholds are set too high, none of the light peaks will be identified. If the level of the background illumination is too high, or contains stray peaks of light, the light peaks from the pupil and CR will be submerged into the noise of the background and the pupil and CR may not be recognizable by the software. If the discrimination thresholds are set too low, the software will pick out areas from the background illumination instead of the targeted peaks of illumination, and pupilary and CR discrimination will be unstable.

67 Figure 6-3. Setting Threshold Discrimination for Pupil and Corneal Reflection (CR). The threshold information shown in Figure 6-3 cannot be displayed by the Eye Tracker program. It would make a useful addition to a later version of the program to be able to display this information on a subsidiary Eye Tracker screen. Illumination is set by first turning Illumination on from the Eye Tracker main screen and raising the illumination level using the Illumination slider. Illumination level can also be controlled with the Pg Up/Pg Down keys. The illumination should be raised about half-way. A circle of light will be visible in the camera wide angle view as the illumination is raised. Threshold levels for pupil and CR edge detection are adjusted with the slide switches labeled Pupil and CR, under Discrimination in the Eye Tracker main window (Figure 4-2 and 4-6). The discriminator settings are shown by the slider positions. Setting the discriminator at the far left position raises the threshold line in Figure 6-3 toward the bright portion of the field. No light peaks will rise above the threshold level and no edges will be detected. Setting the discriminator slider at the extreme right lowers the threshold level toward the dim portion of the field. Low level, stray light peaks will rise above the threshold and dim edges, or noise from the background, will be detected. The Pupil and CR sliders can also be moved using the keyboard. The Up and Down arrow keys move the Pupil slider; the Left and Right arrow keys move the CR slider. Proper pupil and CR discrimination as seen on the video Eye Monitor (not the Eye Tracker main screen) is shown in Figures 6-4 and 6-5. A white circle designating the pupil outline, and a black circle designating the CR outline are displaced slightly to the right of the

68 actual pupil and CR features. The white cross hairs, which indicate the pupil center, actually appear at the center of the white discrimination circle rather than at the center of the pupil image. Similarly, the black cross hairs appear at the center of the black circle rather than the center of the CR image. This offset is purely cosmetic and has no effect on point of gaze computation or display. It is caused by a slight time delay between detection of an edge point and display of the corresponding dot on the monitor, and makes the discrimination outlines easier to see. The true feature edge coordinates are being detected and recorded. Pupil diameter is also visible in this image. Figure 6-4. Pupil Outline and Corneal Reflection as seen in Eye Monitor. To achieve proper pupil and CR discrimination, start with the pupil and CR discriminator slide switches all the way to the left and have the subject look at the center of the monitor. It may help to put the nine-point calibration display on the Scene Monitor and ask the subject to look at point number five. Increase the pupil discriminator level by moving the slide switch to the right. White dots will begin to appear on the eye monitor. These represent discrimination points which are high enough to trigger the pupil edge threshold. As the discriminator is turned up further, these white dots begin to form an outline within the pupil. When the discriminator is turned up far enough, the dots will form a circle that circumscribes the pupil. At this point white recognition cross hairs should appear through the center of the pupil and the green Pupil light on the main

69 screen will come on. The circle and cross hairs will actually be offset slightly to the right of the pupil, as shown in Figure 6-4.

70 Figure 6-5. Eye Monitor view Showing Stable Pupil and CR Discrimination. Observe the eye monitor for several seconds to be sure that recognition is maintained even when the pupil is at its smallest. There may be other areas in the eye image which have white discriminator dots, but the pupil should be the only smoothly enclosed area and should have white recognition cross hairs designating its center. The green Pupil light on the Eye Tracker main screen will stay on, and the Pupil outline and cross hairs in the Eye Monitor will remain steady. If the pupil is very dim and difficult to distinguish from surrounding features, move the Illumination slide switch to the right to increase the illumination intensity. This will probably not be necessary unless ambient illumination or the scene display is especially bright. If illumination intensity is increased, be sure not to make the pupil as bright as the corneal reflection (CR). It is important that the CR remain visible within the pupil image. If the pupil discriminator is too far to the right, other areas may be mistakenly recognized as the pupil and the recognition cross hairs may jump to these areas. Should this happen, lower the pupil discriminator setting by moving the slide switch to the left. If the white circle does not remain solid, increase the discriminator level (move the slide right), even if this means a few white dots appear in other areas. If the pupil discriminator is too far to the right, the cross hairs and all the white dots may disappear (an edge may be detected about the entire monitor screen, and all the white dots will be hidden behind the monitor bezel). Small adjustments of the discriminator are probably best made by clicking the right or left arrow buttons at either side of the slide. This allows finer control than dragging the slider. With the subject looking straight ahead at the center of the monitor, move the CR slide switch to the right until a black outline forms about the CR and black recognition cross hairs designate the center of the CR (the circle and cross hairs will actually be offset slightly to the

71 right as shown in Figures 6-4 and 6-5). If the CR discriminator is too far to the right, the pupil recognition cross hairs as well as the CR recognition cross hairs may disappear. Allow the subject to look about the field and be sure the CR recognition is stable. The green CR light on the Eye Tracker main screen will stay on, and the CR image and cross hairs in the Eye Monitor will remain steady. If black dots form about more than one geometrically satisfactory corneal reflection, the computer software will select the one closest to the pupil for recognition. When the pupil and CR are properly recognized for a given eye image, as shown in Figure 6-5, set the Pan-tilt Tracking button at the bottom left of the main screen to Auto to enable automatic pan-tilt tracking. If using a magnetic head tracker, be sure the MHT has been enabled and the sensor to eye offset has been set, as described in Section 6.2.3, before setting the Pan-tilt tracking mode to Auto. Whether the magnetic system is active or not, once tracking is switched to Auto, as long as a valid pupil is recognized, the pan-tilt camera should be able to follow slow head motions. Auto-focus should keep the eye image in proper focus over a range of several inches (the depth of field will vary from about 6 inches at high zoom and close distances to 10 or 12 inches at slightly lower zoom and longer distances). If the focus range seems asymmetrical (auto focus can follow further in one direction than the other) zoom in very slightly by moving the Zoom switch on the Pan-Tilt/ Setup window to the right to increase the close focus range, or zoom out very slightly by moving the Zoom switch on the Pan-tilt/ Setup window to the left to increase the far range. The Model 504 optics module shutter speed, video gain, and iris (lens aperture) opening can be adjusted from the Pan-Tilt/ Setup window if necessary to achieve the best eye image (Figure 5-6). The default values are no shutter (long open shutter duration), maximum video gain, and maximum iris opening. It may be necessary to adjust these parameters if proper pupil and CR discrimination cannot be achieved using the sliders in the Eye Tracker main window. A common problem occurs when, with the Illumination slider set at the lowest level, the pupil is still so bright that the CR cannot be distinguished. When this happens, any one of these parameters can be altered to dim the entire image, thus allowing proper CR discrimination. If satisfactory pupil and CR images are obtained without using these adjustments, iris, gain and shutter settings should be left in the default positions. Smaller iris values (moving the iris slide toward the left) will make the image appear dimmer but with sharper focus and depth of field, while larger iris values (moving the slide toward the right) will make the image brighter. Lower gain values (moving the slider to the left) will make the image appear dimmer while higher gain values will make the image brighter. Shorter shutter duration (moving the slider to the left) will make the image appear dimmer (although moving images will be crisper ), while longer shutter duration will make the image appear brighter Controlling Ambient Illumination A minimum pupil diameter of about 3 mm is required for successful operation of the Eye Tracker. High levels of ambient illumination which cause excessive narrowing of the pupil should be avoided.

72 If the ambient illumination is too high, valid pupil registration may be difficult to achieve for another reason. As the Pupil discriminator is raised (threshold is lowered), speckling or flaring may appear around the subject s eye (Figure 6-6). If this white glare is slight, a valid pupil may still be acquired, but in some circumstances the glare appears just as pupil discrimination is achieved. When this happens, registration of both the pupil and CR will be unstable. Adjusting the iris and shutter speed via the Pan-Tilt/ Setup window (Figure 5-6) may help some, but a more drastic approach to controlling ambient illumination may be needed. Figure 6-6. Speckling or Flaring around the Eye. The problem is the IR beam is not a perfect pinpoint of light falling on the retina. It casts a circle of light on the subject s face, so the two light sources, reflected light from the face and pupil reflection, increase and decrease together. As the Illuminator level is raised from the Eye Tracker main screen, the light reflected from the pupil and the light reflected from the subject s skin increase in about equal amounts, and separating the two becomes nearly impossible. The solution is to get rid of as much ambient light as possible. As the level of background illumination drops, discriminating the pupil from the background becomes easier and valid pupil registration may be achieved. Overhead florescent lights are often a problem, since they fall on the subject s face and light reflected from the skin will cause the flaring. In this case, turn the lights either down or off. A dimmer on the room lights may be helpful. Darken the windows and shut the door to the lab to block light from outside the room. Never permit direct sunlight to fall on the subject s face, and avoid bright light sources directly over the subject s head or next to the scene being viewed. With several monitors running, there will be enough light in the room to see by. A small table or floor lamp with an incandescent bulb may be placed behind the subject to give some room illumination.

73 Occasionally a subject will be wearing makeup with a metallic base, or the makeup will have metallic fragments embedded in it to create highlights. Ask the subject to come back another time when they are not wearing makeup Controlling Monitor Brightness Flat panel displays emit more light than older displays based on a CRT tube. The light emitted by the monitor shines directly on the subject s face and this light is reflected from the skin, causing the speckling to appear. Use the monitor s built-in menu to lower the brightness of the display. Some experimentation will be required to find a low setting which nevertheless preserves the color relationships in the stimulus material. An anti glare shield on the Display Monitor may be helpful. Eschewing the flat panel display and using one based on a CRT may help avoid this source of problems. You may wish to consult ASL technical support for additional information or for questions about a specific environment Controlling the Brightness of Stimulus Materials If a series of slides is being presented on the subject s display monitor, some slides will be brighter and will throw off more light than others. It may be possible to calibrate the subject successfully using the nine-point display and later find that pupil registration is lost on some slides with particularly bright backgrounds. It is acceptable, even desirable to adjust the pupil and CR sliders on the fly during an experimental session, but it will be easier in the long run to adjust the brightness of the slides beforehand so they are all of about equal brightness. In the Corel Photo Paint program, for example, once the slide is in the work area, opening the Image pull-down menu and selecting Histogram gives a reading for number of pixels in the image. The brightness can be adjusted from within the program and the new version saved. This is not an absolute measure, but it can be used as a relative measure to be sure all of the stimulus slides, including the nine-point calibration slide, have about the same brightness. In one experiment, we found the nine-point calibration slide (see Figure 5-1 and Section 6.4, below) was brighter than the stimulus slides. Subjects were being calibrated under brighter illumination conditions than they were under while watching the stimulus slides, resulting in some instability in discriminating the Pupil and CR signals. A dimmer version of the ASL ninepoint calibration slide helped control some of these problems. See the C: drive on the Display Computer in C:\Photos\NewCal.bmp. 6.4 Subject Calibration This section assumes the default nine-point calibration system is being used with the default settings. Custom calibration options are discussed in Section Two point calibration is discussed in Section A 17-point calibration option is discussed in Section The scene display may be thought of as a field with X and Y dimensions. Each of the nine calibration points has a location within this field. Having calibrated the target points (Section 5.1.2), the program knows the X-Y coordinates for each target point. But it does not know where the line of gaze is directed. The purpose of calibrating the subject is to inform the program about the relationship between these two systems of measurement so that one can be translated into the other.

74 The raw data measured by the Eye Tracker is the separation between the center of the pupil and the corneal reflection (labeled PCR in Figure 9-1). The raw data yields information as to the Horizontal and Vertical orientation of the eye. The program must translate this H-V measurement into a corresponding fixation point in the X-Y field. The relation between the raw separation values and the eye fixation point in the X-Y field will differ for each subject, so every subject must be calibrated individually. The objective is to have the subject fixate on each of the nine calibration points and inform the program as each fixation point is achieved Nine Point Calibration The same nine-point calibration target screen is used to calibrate the subject as was used previously to calibrate the target points in Section (Figure 5-1). There is a nine-point calibration screen in C:\Photos\ASLsampleCal.bmp on the C: drive in the Display computer. It can be displayed on the subject s monitor screen using the XnView program. There is a shortcut icon on the desktop of the Display Computer that starts XnView. After the slide is displayed on the screen, setting View (ALT, V) to Full Screen will make the borders disappear and the slide will fill the screen. The following steps assume that the subject is looking at the nine-point calibration display and that stable pupil and CR discrimination have been achieved. Bring up the Eye Calibration window by selecting Eye Calibration from the Calibrate pull down menu, or by clicking on the eye calibration shortcut button (a nine-point pattern on a white background - Figure 6-7). The Eye Calibration window will open, as shown in Figure 6-8. Figure 6-7. Eye Calibration Shortcut Button.

75 Figure 6-8. Eye Calibration Window. Watch the Eye Monitor or the green Pupil and CR lights on the main screen during calibration of each point to be sure that pupil and CR discrimination are stable before storing the data. Tell the subject to look at point one, as prompted on the Eye Calibration window. Click the Store Data for Current Point button. When Store Data for Current Point is clicked, the value for point number will increment automatically. If preferred over the mouse, use the up or down arrow keys to highlight the Store Data for Current Point button and press <Enter> to store data. To enter points out of order, or to repeat points already entered, use the up and down arrow buttons in the Eye Calibration window to select individual target points. Repeat the same sequence for target points two through nine. When the Eye Calibration window is active, the Scene Display on the Eye Tracker main screen and the display in the Scene Monitor both show the location of the next target point to be entered and not the subject s point of gaze. When point nine is entered, the Eye Calibration window will automatically close and calibration computations will be made and stored. To close the window without computing new calibration coefficients or saving the results, click Cancel. To confirm the accuracy of the calibration, ask the subject to look at each target point again and note the position of the line of gaze cursor or cross hairs on the Scene Monitor. Each target point fixation should be correctly indicated on the Scene Monitor by the line of gaze cursor or cross hairs falling within about one degree visual angle of the target point. Target points that are not correctly indicated on the Scene Monitor must be re-calibrated. Faulty calibration is often caused by attempting to enter Store Data for Current Point when the pupil or CR are unstable. It is important to watch the eye monitor, or the green Pupil and CR lights on the main screen, to be sure that stable recognition cross hairs continue to indicate the pupil center and the center of the corneal reflection. If not, appropriate corrections to the discriminator settings should be made. The discriminator settings can and should be adjusted during the calibration procedure as necessary. If one or more target points are not accurately recorded, reopen the Eye Calibration window using the Calibrate pull-down menu or the Eye Calibration shortcut button. Use the arrow buttons in the Eye Calibration window so that the desired point is displayed. Have the subject look at the point and enter the data by clicking on Store Data for Current Point. If this

76 is the last point to be re-calibrated, click OK to exit. The calibration procedure works best when performed rapidly. If the procedure takes too long, subjects become fatigued and have difficulty maintaining accurate fixation on the target points, or may blink excessively. With some practice, it should be possible to perform the calibration in less than thirty seconds Custom Calibration Options In addition to the standard nine point calibration, the interface program allows the user to customize the number of calibration points and specify certain criteria concerning the calibration computations. This option may be used to clean up the sample points for H and V used in establishing the location of each of the nine (or fewer) calibration points. The default definition of Spread used by the Eye Tracker program is a very stable measure of central tendency and does not normally need to be changed. To access the Custom Calibration Options, check the box labeled Custom Calibration in the Eye Calibration pop up menu. The extended Eye Calibration window will open (Figure 6-9). Unlike the standard Calibration procedure which terminates automatically, once this box is open, the user must click OK to complete the calibration computations and exit the Eye Calibration menu. Figure 6-9 shows the nine target points with a check box next to each point. For each target point, the column for Spread H (horizontal) and Spread V (vertical) shows the interquartile difference in the distribution of 10 sampled points for that target. It is assumed the subject is gazing steadily at the target point during calibration. When Store Data is clicked, ten successive samples of eye position data are taken. Under the default settings, all ten samples are then averaged to give the H-V measure which corresponds to the target point. The user may specify, under Averaging, some limitation on which of the ten samples will be used to compute the mean H and mean V values corresponding to the location of the X-Y point. Selecting the Use all radio button tells the program to use the mean of all 10 sampled values to establish the location of the target point. This is the default. Selecting the Trim 10% radio button discards one of the samples, and selecting Trim 20% discards two of the samples before calculating the mean values.

77 Figure 6-9. Custom Eye Calibration Window. The user may specify which of the target points are to be included in the calibration under Selection Method. In Figure 6-9, there is an X next to each of the nine target points. This indicates that all nine target points will be included in the calibration. Selecting All of the target points is the default. If Best is used as the selection method, the user must specify the number of target points (N) to be used. Those N points are selected where the sum of the interquartile differences for both Horizontal and Vertical is smallest. If Spread is selected, the user must specify a standard deviation (s.d.). Only target points which simultaneously result in a standard deviation for Horizontal and Vertical less than

78 or equal to the standard deviation specified by the user are accepted for calibration. If Manual is selected, no rule is applied. The Manual button allows the user to specify a calibration using any number of target points between three and nine. The user can select the number of points by clicking on the check box for the points to be included. At least three points must be selected. If this condition is not met, a message box indicating the error will be shown and the calibration dialog box will not close. If the Selection Method is changed during calibration, all the target points will be requalified using the Selection Method chosen. An example may help clarify these calculations. The table below shows hypothetical data for pupil to cornea separation (PCR) for one dimension (ten samples). PCR-H raw data Ranked data The data in the first column is the hypothetical raw data from ten successive samples for a single target point. Only the data for H is presented. It is assumed in this discussion that both H and V are extreme when a sample is excluded from the calculations. The data from column one is presented in rank order in the second column. The median is defined as the middle value in the second column. In the case of an even number of data entries, the median is one-half of the distance between the two middle values. In this case: Median = ( ) / 2 =.870 To trim one value, the most extreme value must be found by subtracting the highest and lowest values from the median, and, ignoring the sign, discarding the one which is furthest from the median: = =.030 If the data for V in this sample is also extreme,.840 (H) and the corresponding V will be discarded and the nine remaining samples will be used to calculate the mean PCR values for H and V. To trim two values, the next-most-extreme value must be found:

79 = =.025 The next-most-extreme value is.845, which will also be excluded. The remaining eight samples will be used to calculate the mean PCR for H and V. The data can be divided into quartiles (quarters). The quartiles are designated Q1, Q2, and Q3. Q1 marks the point below which 25% of the cases will lie. It is the case which splits the bottom 50% of the cases in half. Q2 marks the point below which 50% of the cases will lie. Q2 is the same as the median. Q3 marks the point below which 75% of the cases will lie. It is the case which splits the upper 50% of the cases in half. In the data in the second column, Q1 =.858 and Q3 =.879 The interquartile difference is defined as Q3 - Q1: Q3 - Q1 = =.021 When Best is used as the Selection method, the program attempts to find the N target points for which Q3 - Q1 is minimal for both Horizontal and Vertical. That is, the sum of H(Q3- Q1) + V(Q3-Q1) must be minimized. When Spread is used as the Selection method, only those target points are used which have s.d.(h) and s.d.(v) equal to or less than the specified s.d. In the hypothetical data given above, the mean (H) =.867 and s.d. (H) = Two-Point Calibration If a 2-point calibration is preferable, the interface program has a separate menu item, Quick Calibration under the Calibrate pull down menu. Quick Calibration assumes that target points will be displayed in the upper left and lower right corners of the display (points 1 and 9 where point 1 appears in the upper left hand corner and numbers increase to the right ending with point 9 in the bottom right corner) The 17-Point Calibration Option If the scene image is significantly distorted (e.g., an image from a scene camera with a wide angle lens that produces fish eye distortion), or the eye feature to point of gaze relation is especially complicated for some other reason, the 17 point mode may improve accuracy. The 17-point calibration mode uses 17 calibration target points instead of the usual nine to map the relationship between the pupil and corneal reflection (PCR) features on the eye with target points on the scene. If after doing a normal 9-point calibration it is found that data is accurate at the normal 9 calibration points, but inaccurate when the subject is looking between the locations of the normal 9 points, then the 17-point mode will probably improve results. To enable the 17-point mode, check the 17 point eye calibration check box on the System Settings dialog window, under the Configuration pull down menu (Figure 4-3). Target points should be arranged about half way between the outer points of the normal nine point display and the center point, as shown in Figure 6-10.

80 Figure Seventeen Point Calibration Target. Procedures for setting target points and performing the eye calibration are the same as those described above, except all 17 points must be designated by the Set Target Point procedure, and the subject must look at all 17 points during calibration. The Eye Calibration window will automatically close, and the computations are performed, after the 17 th point is entered. 6.5 Manual Eye Position Offsets If the optics module is bumped, or calibration shifts slightly for some other reason, the user may add offsets to the final calculated position of the target points from within the Eye Tracker program without repeating the subject eye calibration. There are two procedures that can accomplish an offset. Display the nine point calibration target screen. Pull down the Calibrate menu from the main screen and select Eye Position Offset (Figure 6-11). Have the subject gaze at the point indicated in the Set Point box and use the arrow buttons to position the cursor over that point in the Eye Monitor. Exit the offset mode by clicking Close.

81 Figure Eye Position Offset Window. Alternately, ask the subject to fixate on point number 5 on the calibration screen, select Quick Offset from the Calibrate menu, or select the Eye Position Offset pop up window and click Set point 5. The computer will automatically calculate the appropriate offset values both vertically and horizontally to map the current gaze point to target point 5. To reset both the horizontal and vertical offsets to zero (back to the original subject eye calibration), select Reset Offsets directly from the Calibrate menu, or click on Reset from the Eye Position Offset window. Offsets are automatically set to zero when the subject eye calibration sequence is started. 6.6 Saving and Loading Calibration Files Calibration and target point values are automatically saved to the default calibration file, e5000.cal, whenever subject calibration or set target points procedures are performed. The current values can be saved to a file of the user s choice at any time with the Save Calibration selection under the Calibrate menu (Section 4.2.3). To load target point and calibration data from a previously saved file, use the Read Calibration selection from the Calibrate menu, also described in section The calibration files save only target point and calibration data. Offset values are not saved. Current offsets, if any, are automatically zeroed whenever a calibration file is loaded.

82 Chapter 7: Eye Movement Monitoring The operator should look at the Eye Monitor throughout an eye monitoring session to make sure the pupil and CR are being correctly recognized (Section 6.3). The operator should also watch the Scene Monitor point of gaze display to be sure that the data makes sense. The operator can select a white or black cursor or set of cross hairs as the point of gaze indicator by using the check boxes under POG Indicator on the left side of main Interface screen (Figure 7-1). Digital eye position values are displayed continuously on the Interface screen Scene POG for both H and V. Beneath that, pupil diameter is also displayed. Data can be recorded as described in the next section. The settings described in this chapter are only used when data is being recorded by the Eye Tracker program on the Interface PC. If Gaze Tracker is run on the Display PC, the Eye Tracker program communicates the raw data to Gaze Tracker, which records and manipulates the data. See Chapter. Note that using one program to record data does not disable the other, or prevent it from also recording and storing data. 7.1 Blink Handling Figure 7-1. Recording from Eye Tracker Main Screen. A setting in the e5000.cfg file (in the same directory as e5win.exe) controls the behavior of the cursor in the Scene Monitor during blinks. Under the default settings, if the subject s eye closes, or if pupil recognition is lost for some other reason, the scene POG cross hairs or cursor freezes at the current position until 12 samples are taken (200 msec at 60 Hz). The cursor then

83 jumps to a default position of (0,0). The cursor is not visible in the default position and will seem to disappear. This prevents the cursor from disappearing during eye blinks, which are usually shorter than 200 msec, but makes the cursor disappear when the pupil is lost due to extended eye closure, poor system adjustment, etc. The recorded gaze coordinate values also freeze, as described above, and then switch to (0,0). The value recorded for pupil diameter is always zero for any sample in which a pupil is not recognized. This feature can be modified to change the number of fields during which gaze coordinates freeze, or can be disabled altogether so that the cursor will jump to the default position (seem to disappear) immediately upon loss of pupil recognition. To modify this feature, open the e5000.cfg file with a text editor and find the line that reads : eye_position_blink_filter_value=12 and change the 12 to the desired number of fields. To disable the feature, replace the 12 with 0. Be sure to save the modified file as e5000.cfg. The next time the Eye Tracker is run, or the next time Calibrate/ Read Calibration/ e5000.cfg is selected, the new value will be used. Note that if using a 50 Hz eye camera (PAL format), 12 fields correspond to more than 200 msec, and it may be desirable to change this value to 10 fields. If using an optional highspeed camera, the value must be increased proportionately if it is to correspond to about 200 msec. At 120 Hz, the corresponding value would be 24 fields, and at 240 Hz the corresponding value would be 48 fields. 7.2 Data Recording Eye position vertical (Y) and horizontal (X) coordinates, pupil diameter, the state of the 16 bit parallel port on the 5000 Controller, called XDAT (Section 7.2.4), and event markers entered from the keyboard can be recorded on the Interface PC hard disk. A field of data, consisting of the elements just listed, is recorded every 60th of a second (60 Hz update rate). If averaging has been specified on the System Settings window (Section 4.2.2), the averaged gaze coordinate (X-Y) data is recorded. After a period during which a pupil was not recognized and no valid gaze measurement could be made, the first valid measurement is not averaged. The next measurement is averaged with the previous valid field, and the number of fields averaged increases in this way until the specified value is reached. Only gaze coordinates are averaged. Neither pupil diameter, nor any other recorded values are averaged Open A New Data File To open a new data file, select New from the File pull-down menu, or click the new file icon on the shortcut bar (Figure 7-1). Use the standard file browser and dialog pop box up to specify a directory and file name. The default extension for a data file is.eyd. Once a file is opened, the file name will be displayed on the interface program screen just above the comment field. Initially the file name will be displayed in black letters with the message (Paused) after the file name. After opening a file, type any desired text in the Comment field on the interface program screen and the text will be saved as part of the data file.

84 7.2.2 Start Recording Click the record icon (right arrow icon on the shortcut bar), press the R key, or select Start Recording from the File menu. The file name will change to red letters and the message following the file name will change to (Recording). The disk bytes free indicator will also change as disk space is used up Event Markers Any one of 10 manual Marker Flags may be added to the data by clicking one of the numbered Mark buttons (icons labeled 0 through 9) on the shortcut bar, or by pressing the corresponding number key XDAT External Data A 25-pin female D connector labeled XDAT is provided on the back panel of the model 5000 Control Unit. The XDAT connector provides access to a parallel port that may be used to input data from an external source. Sixteen bits of parallel, TTL level, positive true data may be recorded. Since there are 16 data bits, there are 65,536 unique configurations that might be applied to the port to indicate external events the user may wish to record as part of the data stream. The state of the XDAT port is recorded for each sample point. The pinout specifications for the XDAT connector are listed below. Each bit is interpreted as 0 when the corresponding pin is low (ground), and as 1 when the corresponding pin is high (+5 Volts). The Eye Tracker samples the XDAT port once every eye camera video field; the data is recorded along with the eye position data from the corresponding eye camera video field. The current decimal value of XDAT is displayed near the bottom center of the interface main screen. Table 5-1. XDAT connector FUNCTION PIN NUMBER (25 pin female D type ) XDAT bit 0 (LSB) 2 XDAT bit 1 3 XDAT bit 2 4 XDAT bit 3 5 XDAT bit 4 6 XDAT bit 5 7 XDAT bit 6 8 XDAT bit 7 9

85 XDAT bit 8 10 XDAT bit 9 11 XDAT bit XDAT bit XDAT bit XDAT bit XDAT bit XDAT bit 15 (MSB) 1 Ground 18,19,20,21,22,23,24, Auto Eyedat When Auto EYEDAT is enabled on the System Settings dialog window (Figure 4-3), data recording is controlled by the most significant bit on the XDAT port (XDAT bit 15, which is pin 1 on the XDAT connector). To enable Auto EYEDAT, pull down the Configure menu and select System Settings. Check the box labeled Auto Eyedat. To record data using the auto EYEDAT function, first open a data file in the usual way. Once Auto EYEDAT is selected, the Record and Stop buttons on the shortcut bar, as well as the Record and Stop selections on the File pull down menu will be gray and inoperative. If XDAT bit 15 is 0 (ground), the Paused message will be displayed next to the file name to indicate that no data is being recorded. When bit 15 is set to 1 (+5 V), recording of a data segment will begin, and the message next to the file name will change to Recording. When bit 15 is returned to 0, recording stops, and the Paused message will reappear. Setting or resetting the most significant bit of XDAT is equivalent to clicking Record or Stop in the normal operation mode. When the Auto EYEDAT check box on the System Settings dialog window is not checked, record and stop commands operate normally on the Interface program. In this case the XDAT value has no effect on starting or stopping data recording. The state of the XDAT port is recorded for each sample, along with other data, whether or not the Auto EYEDAT feature is being used Stop Recording To stop recording, click the Stop Recording icon (Black Square) on the shortcut bar, press the S key, or select Stop Recording from the File menu. The file name will change back to black letters and the (Paused) message will reappear. A Data Segment number is recorded as part of the data file. Each time the Start/ Stop sequence is completed, the Data

86 Segment number is incremented. A maximum of 169 data segments may be recorded Close Data File To close the data file, select Close from the File pull down menu or click the Close Data File icon. The data is saved in a binary format. If Save as File Type is left as Eyedat (the default) the data file will be saved with an.eyd filename extension. This is the recommended procedure. The Eye Analysis program, Eyenal, by default looks for data files with the.eyd extension. 7.3 View Recorded Data A file viewer is provided in the Eye Tracker interface program as a means of quickly checking the raw data, to be sure something has been recorded. To use the viewer, pull down the Misc menu and select View Data File. Click the Browse button (3 dots on a gray field) next to the Data File box. Select the desired file, and click Open. If a file is currently open, it will automatically be loaded into the viewer. It may take several seconds to load, depending on file length. Loading the file can be aborted by clicking the Close button at the bottom of the viewer. Once the file has loaded, file header information is displayed in the top section of the viewer, and data is displayed in labeled columns in the scrolling window in the bottom section of the viewer. Use the standard scroll bar to scroll to move through the data. 7.4 Working with EYEDAT files Recorded data can be accessed off line by: using a set of C routines, provided by ASL, within user-created software to read the binary data files; using a binary file description, provided by ASL, to write access software from scratch. Source code for the C access routines and the binary file description are provided in a file labeled ACCESS on the software CD; using ASL s EYENAL data analysis program; converting the files to a simple ASCII format using the CONVERT function in the EYENAL data analysis program. EYENAL is described in a separate instruction manual Vertical and Horizontal Eye Position Coordinates The eye position coordinates can be thought of as point of gaze coordinates on the scene monitor screen or on the Interface program Scene POG display. The coordinates are given in Eye Tracker Units (ETU s) and are independent of the hardware being used. The coordinate frame origin (0,0) is in the upper left corner of the monitor screen. Horizontal position values increase to 260 at the right edge of the monitor, and vertical values increase to 240 at the bottom of the monitor. Note that it is possible to have negative data values or values

87 that exceed the maxima just cited, since the subject may be looking beyond the extent of the scene monitor field Conversion of Pupil Diameter to Metric Units Once pupil discrimination has been achieved, pupil diameter is displayed in Eye Tracker Units (ETU s) on the Eye Tracker main screen. A zero pupil diameter value indicates that no pupil was found for that field. This may be due to an eye blink, an incorrect discriminator setting, or some other occlusion of the pupil. Pupil diameter values are not scaled to metric units. The scale factor depends on the magnification of the eye camera lens and the eye to camera distance. The scale factor is usually very close to 0.15 millimeters/eye-tracker-unit. Section 7.2(2)(j) of the EYENAL manual explains how to convert pupil diameter units as reported by the Eye Tracker into millimeters. The model eye may be used to find the conversion factor. Fasten the model eye so that the white 4mm circle is at about the same distance from the optics as the subject s eye. Turn on the illuminator, aim the pan-tilt camera at the model eye, and adjust the discriminator settings to obtain discrimination outlines and center cross hairs (Section 6.3). Note the pupil diameter value on the Eye Tracker main screen in the Pupil Diam box. To compute a scale factor, divide 3.96 by this value. Convert the displayed or recorded pupil diameter values to millimeters by applying this scale factor (value in millimeters = scale factor * recorded value). Scale Factor = 3.96 / Pupil Diam in ETU s Then for any subject: Pupil diameter in mm = Pupil Diam in ETU s * Scale Factor The Scale Factor can be measured once at the beginning of an experiment. It does not have to be re-computed for each subject. So long as the subjects in an experiment are seated in approximately the same position at about the same distance from the eye camera, minor intersubject differences will not affect the accuracy of measurement of pupil diameter Magnetic Head Tracker Data in X-Y-Z Dimensions If a magnetic head tracker (MHT) system is connected and enabled, MHT data will be recorded along with Eye Tracker data. The MHT data consists of X, Y, and Z position values, and azimuth, elevation, and roll orientation values. These values refer to the position and orientation of the MHT sensor with respect to the magnetic transmitter. MHT data recorded by the Eye Tracker program is recorded using the same binary scaling as the MHT device. After retrieving these values as integers, with data access routines provided by ASL they can be converted to meaningful units using the scale factors listed below. MHT type position scale factor (inches) position scale factor (centimeters) angle scale factor (degrees) Ascension Bird or Flock Polhemus

88 FASTRAK Polhemus 3Space ISOTRAK or The Sample as a Sliding Average If averaging has been specified in the System Settings window, the averaged gaze coordinate data is recorded. After a period during which a pupil was not recognized (no valid gaze measurement could be made) the first valid measurement is not averaged. The next measurement is averaged with just the previous valid field, and the number of fields averaged increases in this way until the specified value is reached. Only gaze coordinates are averaged. Neither pupil diameter, nor are any other recorded values are averaged An Example of Data Output The binary data file can be read by the EYENAL program and output in ASCII format. See the Eyenal manual, or download the file Eyenal_Manual.pdf from ASL. The following is an example of an ASCII data file produced by the Eye Tracker program. The first section is a header containing comments entered by the operator: EYEDAT V1.20 Recorded: December 9, 1997 Time: 18:31:3.12 System: 5000 Update rate: 60 Hz Number of user segments: 1 Demonstration User Description... Demonstration User Description... A "segment" is all the samples collected between "Start Record" and "Stop Record." If recording is started and stopped several times in one session, the data for one subject might have up to 169 Segments. Time is the time recorded from the system clock on the Interface Computer. Segment: 1 Start: 18:31:16.36 Stop: 18:31:53.90 Here's the raw data. If the data was exported as an ASCII file to be analyzed by an external statistical program, the user would remove everything down to the dashed line ---- Fld Seg Time Total_Secs H-pos V-pos Diam Xdat Mark :31: :31: :31: :31: :31: :31: (snip) :31: :31:

89 :31: Fld = Sample Number. Sample rate = 60 Hz. Seg = Segment number (only one segment in this example). Time = Local system clock time in 24-hr format. Seconds to three decimal places. Total Secs. = Total seconds since the previous midnight. H_Pos = Horizontal gaze position in Eye Tracker Units (ETU's). V_Pos = Vertical gaze position in ETU's Eye Tracker Units can be converted to inches, centimeters, or angles. The Eyenal program can be used to calculate fixations within each "Area of Interest" (AOI) defined by the user, how much time is spent within each AOI, etc. Since raw data is recorded here, new AOI s can be defined by the user after the data has been collected. Each subsequent analysis by Eyenal spawns a new data file specific to the type of analysis being conducted. Diam = Pupil diameter in Eye Tracker Units. Must be converted to mm. If the person blinks or looks away and no data can be acquired, Diam = 0. XDat = state of the XDAT port on the Control Unit - not used in this example. Mark = Event Marker inserted by clicking on Eye Tracker interface main screen - not used in this example. If an MHT is being used, the data file will also include the following columns: MHT-X, MHT-Y, MHT-Z, Azimuth, Elev (Elevation), and Roll. These refer to the MHT sensor orientation within the magnetic field created by the MHT transmitter. 7.5 Serial data output Eye tracker data can be output through an RS232 port, labeled Serial Out on the Control Unit. The port is set to baud, 8 data bits, 1 stop bit, no parity. Other baud rates are also possible (consult ASL for details) Interface Cable The Control Unit Serial Out connector is a 9-pin male D-type. Only Transmit, Receive, and Ground lines are used: Serial Pin Out Signal 3 Serial data from host to Control Unit 2 Serial data from Control Unit to host 5 Ground An example is shown below of wiring for a cable to connect the eye tracker Serial Out

90 port to a standard 9-pin COM port on a PC. 9 pin female 9 pin female Serial Protocol and Data Format The serial data output port can be set to use either a demand mode or a streaming mode. In the demand mode, the host computer requests a data field by transmitting a single byte of any value. In response, the Eye Tracker transmits a field of data. After a data request is received from the host, the Eye Tracker PC will begin to transmit the requested field within one update interval. In the streaming mode, no data request is required. Data will continually stream from the Serial Out port. The data is encoded, however, so that the first byte of a data field can be identified. Encoding of the data field is shown below. Note that the most significant bit of the first data field byte is always set (1). The most significant bit of all other bytes in the data field are always reset (0). For the standard data set, the encoded data field is 10 bytes long rather than 8 bytes. The host computer must find 10 sequential bytes starting with a byte whose most significant bit is 1, then decode the data by reversing the encoding process. Sample source code for decoding streaming data can be provided by ASL upon request. After decoding, the standard data buffer consists of the 8 bytes listed in Table 6-1. Table 6-1. Standard Serial Out Data Field Byte Description 1 Status (0 = normal, >0 = error condition) 2 Pupil diameter, most significant byte (0=loss) 3 Pupil diameter, least significant byte 4 <Used only by model 501 system with EYEHEAD Integration> 5 Point of gaze horizontal coordinate most significant byte (scene monitor coordinates)

91 6 Point of gaze horizontal coordinate least significant byte 7 Point of gaze vertical coordinate most significant byte 8 Point of gaze vertical coordinate least significant byte Note that if using the streaming mode, the above list shows the data after decoding. Each coded data field read by the host will consist of 10 bytes. The system can also be set to provide alternate data buffer contents. The buffer contents and the output mode (demand or streaming) are controlled by a value in the e5000.cfg file. View e5000.cfg (in the same directory as e5win) with a text editor and find the line that reads: serial_data_output_format_type= For the standard 8-byte output buffer in demand mode, set the value after the equal sign to 1. For the same buffer contents in streaming mode, set the value to 129. For other alternatives, consult ASL. Resave the file as e5000.cfg. 7.6 Analog Data Output Two channels of analog output are available on the Eye Tracker Control Unit ports labeled Analog I/O. One analog channel outputs a voltage corresponding to the horizontal eye position coordinate, and the other outputs a voltage corresponding to the vertical eye position coordinate. The voltages are proportional to the horizontal and vertical positions of the point of gaze cursor on the Scene Monitor and the recorded gaze coordinates. Pupil diameter is not output, but when no pupil is recognized, the analog outputs are set to a default value. Although the maximum voltage swing on each channel is from 0 to +12 V, when the cursor is within the Scene Monitor display the swing is from +3 V to +9 V. The voltage should exceed 9 V only if the subject looks beyond the part of the scene shown at the bottom or right edge of the scene camera image, or if the system mistakes some extraneous reflection for a pupil or CR and produces nonsense data. Normally, approximately 3V from both channels corresponds to the upper left corner of the scene monitor, and 9V from both channels corresponds to the lower right corner. The center of the scene monitor screen should correspond to about +6V. The analog outputs remain live, showing changing voltage outputs, when the program is in Set target or Check target modes. For example, the Set target function, which allows the point of gaze cursor to be controlled with the mouse, can be used to check the precise correspondence between any point in the scene image and the analog outputs. The analog output reflects the averaging specified in the System settings dialog window, (Section and Figure 4-3), as do the stored digital data and the cursor in the Scene Monitor. The Scene Monitor cursor position as well as the analog values will freeze at their last position during pupil or CR loss for the number of fields specified in the e5000.cfg file ( eye_position_blink_filter_value = ), and will then jump to a default value until the pupil is again recognized. This prevents blinks from causing the values to jump, as explained in Section 7.1. The default for the number of fields during which the value will freeze during losses is:

92 eye_position_blink_filter_value=12. Edit the e5000.cfg file with any text editor to change this to any desired value. Setting the value to zero will cause the values to immediately jump to the default value during any field in which the pupil is not recognized. In other words, if the blink filter value is set to zero, neither the Scene Monitor cursor nor the analog values will freeze at their previous value for any period of time during pupil loss, but will go to the default position for the entire period during which the pupil is not recognized. Note that the blink_filter value is a number of fields, not a time. When running at 60 Hz, 12 fields correspond to about 200 msec, but at 240 Hz, 12 fields only corresponds to about 50 msec. The value must be adjusted, depending on the sampling rate being used.

93 Chapter 8: Theory of Operation and Specifications 8.1 Pupil and CR Recognition The Eye Tracker optics module is designed so that the near infrared eye illumination beam is nearly coaxial with the optical axis of the pupil camera. Because it is coaxial with the light source, the camera lens captures the partially collimated beam that is reflected back from the retina, and the image reaching the camera sensor is that of a back lit, bright pupil. This bright pupil image can usually be much more easily discriminated from the iris and other background than a black pupil image. The amount of reflected light that reaches the camera from the retina is approximately proportional to the fourth power of pupil diameter. Pupil brightness therefore varies significantly with pupil diameter. Even when a subject s pupil is at its largest and brightest, the reflection of the illuminator from the front surface of the cornea (corneal reflection or CR) is normally much brighter than the pupil. Thus the pupil can usually be distinguished from the background and the CR can be distinguished from the pupil on the basis of brightness. When a subject s pupil becomes very small (3 to 4 mm diameter), sections of the eyelid, cheek, or sclera that are also in the camera field often appear as bright as the pupil. In these cases, size, shape, and smoothness criteria must be used to help identify the pupil. In some cases more than one area will be as bright as the CR. If more than one bright spot will satisfy the proper size and shape criteria, the computer selects the spot closest to the pupil center as the CR. Once the pupil and CR are identified, the computer calculates their centers for use in determining eye line of gaze. Note that when the eye looks away from the illuminator more than about 25 degrees, the CR no longer appears on the cornea and cannot be detected. 8.2 Eye Line of Gaze Computation The separation between the pupil and the corneal reflection (CR) varies with eye rotation (change in point of gaze) but does not vary significantly with eye translation (head movement with respect to the eye camera). A change in pupil-cr separation is approximately proportional to the change in point-of-gaze. The precise relationship between eye line of gaze and the pupil-cr separation (PCR) as seen by the camera is diagrammed in Figure 8-1 for a single axis. The relation reduces to PCR = K sin( θ) where 2 is eye line of gaze angle with respect to the light source and camera, and K is the distance between the iris and corneal center of curvature. The corneal reflection (CR) is detectable over about a degree diameter visual angle field. In addition to the geometry described by the above equation, it is necessary to account for intrasubject differences in corneal shape and other second order effects. The Eye Tracker

94 computes eye azimuth and elevation angles as polynomial functions of pupil CR separation along each axis, including cross talk and corner terms. Data stored during the calibration procedure is used by the computer to calculate the polynomial coefficients for each subject. 8.3 Timing Figure 8-1. Relation Between Line of Gaze and Pupil/CR Separation A data sample is output by the Eye Tracker Control Unit for every eye camera video field. There is a transport delay of about 3 video fields, as shown in Figure 8-2. The camera pixels charge up during 1 video field, the video data is transmitted to the system and digitized during the next field, and is processed by the system during part of the third field. The new data is available at the serial or analog output port near the end of the third field, so each data sample contains information that is about 3 fields old. With a 60 Hz (NTSC format) eye camera, this corresponds to a transport delay of about 50 ms (3/60 of a second). With a 50 Hz (PAL format) eye camera the delay is 60 ms (3/50 of a second). With an optional high-speed camera running at 240 Hz, the delay is 12.5 ms. Note that averaging, as specified in the System Settings window (see section 4.2.2), will add lag to the gaze coordinate data, since each sample will be the average of the most recent computation and the specified number of previous computations.

95 Figure 8-2. Eye Tracker Timing 8.4 Specifications We are running Eye Tracker: Version FPGA File = E5121.bxt DSP File = e5364_01.lxr The Controller is: Model 5000 CU Serial Number: 1341 Measurement Allowable eye movement Precision Accuracy Eye line of gaze with respect to a stationary viewing surface. Along the horizontal axis, 50 degrees or more. Along the vertical axis, 35 degrees or more depending on optics placement and eyelids. (Field will generally be oval in shape.) Better than 0.5 degree. Spatial error between true eye position and computed measurement is less than 1 degree. Errors may increase as the head moves significantly from its initial position. Errors may also

96 increase when gaze is beyond the outer boundary of the calibration pattern. Eyeglass and contact lens acceptance Ambient illumination Sampling and output rate Physical dimensions (approx.) Most are accepted. Eyeglasses may need to be tilted with respect to the head if a specular refection from the glasses interferes with the pupil image. Complete darkness to moderate illumination resulting in pupil diameters greater than 3mm. Brighter environments possible with special precautions. 60 Hz (or the country s television scan rate standard). 120 Hz and 240 Hz available as options Control unit is 3.25 h X 10.0 w X d and weighs 4.5 lb. Pan-tilt module is 4 h X 5.5 w X 6 d and weighs 2.75 lb. (note: shape is irregular). Manuals for the 504 system are available from the ASL web site: Login using your address, for example: [email protected] The password is: eyetracker (all lower case). The phone number for Technical Support is: (781) , Ext. 28. Eventually the techsupport site will have a Support Forum where users can exchange ideas and share information, but that part of the site is still under construction.

97 Chapter 9: Trouble Shooting 9.1 General Approach The following information is essential to determining the cause and solution to occasional system problems. In the event that ASL is contacted the basic concepts in this section will facilitate support. The ASL Eye Tracking system integrates several components so the first step in troubleshooting is to determine which sub-system(s) is the cause of the problem. In most cases, the symptom of the problem itself will lead to an immediate determination of the general location of the difficulty. From there a number of diagnostic tests can be undertaken to pinpoint the problem and rule out other possible causes. 9.2 Functional System Description The eye tracker consists of five sub-systems: Optics. This consists of the PanTilt camera and illuminator-ring and the scene camera or scan converter. These are responsible for producing eye and scene video signals that are sent to the Control Unit. Control Unit. The model 5000 Eye Tracker Control Unit supplies power to the eye camera, scene camera, and illuminator; accepts video input from the eye and scene cameras; recognizes pupil and CR features and determines their centers; computes point of gaze; superimposes edge discrimination dots and cross hairs or cursors on the eye and scene video images for display on Eye and Scene Monitors; sends commands to the pan-tilt mechanism; receives command input from the Interface PC; and sends data to the Interface PC. Interface PC. The Interface PC uploads software to the Control Unit; sends commands (through e5win.exe) to the processor to adjust discriminator values and illuminator values, launches the calibration process, and receives data from the Control Unit for display and digital recording. Display. This consists of the black and white Eye Monitor and Scene Monitor. Power. The power supply module receives AC power from a standard outlet ( VAC, 50-60Hz) and supplies 12 VCD to the Control Unit. 9.3 Functional Priorities It is important to understand what functions are dependent on other functions in the operation of the eye tracker system. In this way, time is not wasted servicing or troubleshooting dependent or secondary operations. The following elements of the system must be present in the order shown. If there is a service problem, the top-most one should be approached first. 1) A successful software upload to the Eye Tracker Control Unit, 2) Successful communication between Control Unit and the Interface PC (e5win Interface program indicates online ),

98 3) A good Eye Monitor picture, 4) Pupil outline discriminated on the Eye Monitor, 5) Corneal reflection discrimination outline on the Eye Monitor, 6) Pupil and corneal reflection recognition cross hairs on the Eye Monitor, 7) Pupil and corneal reflection cross hairs on the Interface PC display, 8) Automatic tracking by the pan-tilt mechanism of a properly recognized pupil, 9) A good scene image (if a scene camera is being used), 10) Successful eye calibration, 11) Point of Gaze cross hairs or cursor on the Scene Monitor, 12) Reasonable point of gaze cursor on the Interface PC Scene POG display, 13) Serial Output from the control unit is successfully communicating to an external device (if using the serial output). 9.4 Preliminary Trouble Shooting If there is a problem with the Eye Tracker system, the following steps should be undertaken first: Check all power switches and AC connectors for all individual assemblies. Check all the connections and connectors to be sure that they are all going to the proper points. Check eye tracker functions in the order listed in section 9.3. Contact ASL when one or more of the steps in section 9.3 is not satisfied or for clarification on any issue. 9.5 Trouble Shooting Tips The following are responses to frequently asked questions. Specular Reflection (Glasses). When a subject is wearing glasses, depending on the angle of the Pan-tilt camera and the subject, the ring of LEDs might create a specular reflection. This reflection can result in Pupil and/or CR recognition difficulty. (Note that there are other possible sources of light that cause specular reflection.) When presented with this situation try the following: wipe glasses clean,

99 check environment lighting (window, overhead, display, monitor brightness, etc.), increase magnification. Increasing the magnification will result in the camera s field of view being occupied by more of the eye and less of the surrounding features (e.g. glasses frames), Change the present angular relationship between the subject and the Pan-tilt camera. Try an adjustable height chair. Have the subject raise their chin. Attempt to view subject s eye from a steeper angle (move subject in towards the camera or raise the seat height). Angle the glasses so that the lenses tilt upward and arms of the frame tilt downward. Cover the area of the frame causing the reflection (black tape). Eye Calibration and Pupil/CR recognition. If experiencing trouble with the Eye calibration consider the following: What is the eye-to-camera distance? Optimal performance will be achieved with an eye to camera distance of inches ( cm). Is recognition of the Pupil and CR stable? If the black circle and cross hairs indicating the CR appear to be recognizing the Pupil (i.e. the black circle is jumping up around the Pupil): a) bring the CR slider back to the left (raise the CR threshold), b) reduce the Illuminator level so there is more contrast between the Pupil and CR. Remember that the CR should look brighter than the Pupil. If the white circle indicating the Pupil appears to be caving inward then drag the Pupil slider to the right (lower the pupil threshold). This will allow the system to detect the Pupil when it has dimmer edges. If the white circle indicting the pupil appears to be pulling away from the Pupil, then drag the Pupil slider back to the left (raise the pupil threshold). This will help the system to ignore the background. If the CR is not visible when inside the Pupil: When a very bright pupil (prevalent in younger subject populations) is burying the CR you can increase the contrast between the CR and the Pupil by adjusting the Iris setting. The Iris adjustment can be found in the Setup menu under the Pan-tilt pull down. Moving the Iris slider to the left, toward Min, will make the iris opening smaller, resulting in increased contrast between the Pupil and CR. Target points. Are the Target Points spaced too far apart for the given subject-to-scene distance?

100 Is the visual angle between the each target point greater than 20 degrees? Improperly spaced target points will cause the subject to rotate their eyeball past the trackable +/- 25 degrees and cause the CR to become lost onto the sclera. Try moving the subject away from the plane of the calibration scene, or move the target points inward. You can determine the maximum distance at which the outer target points can be placed by having the subject follow a finger tip from the middle of the calibration scene to its edges and watching the Eye Monitor until the CR is lost. MHT If an error message appears; If communications freeze; If the front light on the MHT Control Unit is blinking (Ascension Flock of Birds only); press <Enter> or click on OK to close the error box. Check all cable connections. If the Polhemus MHT is being used, be sure the dip switches have been set correctly for 19,600 baud. All of the dip switches except #1 should be pointed UP. (There are no dip switches on the Ascension MHT system.) Be sure the correct System Type is selected under the MHT pull down menu (Section 4.2.4) and the correct baud rate (19,600) has been selected (Section 4.2.2). Power off the MHT system, restore power to the MHT system, and try to enable it again (Section 6.2.3). If there is still a problem, consult ASL. 9.6 Using the Model Eye One of the accessories supplied by ASL is a model eye, or target bar, that can be used to simulate the image received from a real eye. It consists of a thin, 2 inch by 6 inch piece of aluminum, painted black, with a white, approximately 4 mm diameter circle, and a small ball bearing. The diameter of the white circle is 3.96 mm. When viewed by the eye tracker optics, the white circle looks like a bright pupil image, and the reflection from the ball bearing looks like a corneal reflection. The model pupil and corneal reflection (CR) images will not mimic the relative motion of the pupil and CR when a real eye rotates. They do, however, provide stationary models that can be used to test eye tracker discrimination functions, to practice discrimination adjustments, and to calibrate pupil diameter. To use the model eye, place it so that the white 4mm circle is at a normal eye distance from the optics, turn on the Illuminator, aim the pan-tilt camera at the model eye, and adjust the discriminator settings to obtain discrimination outlines and center cross hairs for pupil and CR (Section 6.3).

101 To compute a scale factor for pupil diameter values displayed on the computer screen, or recorded by the Interface PC program, first obtain proper discrimination on the model pupil, then note the pupil diameter value on the Eye Tracker main screen ( Pupil Diam ). To compute a scale factor, divide 3.96 by this value. Convert the displayed or recorded pupil diameter values to millimeters by applying this scale factor (value in millimeters = scale factor * recorded value).

102 10.1 Hardware Connections Chapter 10: Gaze Tracker Setup The Gaze Tracker program runs on the Display PC. The connections between the Display PC, the Interface PC, and the other hardware are shown schematically in Figure 2-2. The Monitor output from the Display Computer connects via VGA monitor cable to the Scan Converter. The Scan Converter passes the signal on to: 1) the monitor viewed by the subject, and 2) the ASL 5000 Controller. The controller passes the signal to the b/w Display monitor. It may be convenient to insert a Y-connector between the output of the Scan Converter and the subject Display Monitor, so the scene as viewed by the subject can also be seen on a second monitor, which faces the experimenter. A large screen display facing the experimenter is easier to work with than the small display reproduced on the b/w Display monitor. Com1 from the Display Computer is connected via serial cable to the Serial Out jack on the 5000 Control unit. The Eye Tracker program outputs the data stream over this cable so it can be saved by Gaze Tracker. Other connections for the Interface PC, the Controller box, and the two b/w monitors are described in Chapter Installing the Gaze Tracker Program We are running Gaze Tracker version FULL ASL. This version of the program was created on 6/22/04. Later versions have a serial number, but this one doesn t have one. The Gaze Tracker program installs itself on the Display Computer in the C:\ ERT\ bin directory (folder). All the Eye Tracker manuals and the Gaze Tracker manuals are available as.pdf files on the Display Computer in C:\ ERT\ bin\ manuals>. When the Gaze Tracker program CD is inserted into the drive on the Display Computer, installation should proceed automatically. If the installation program fails to start, run the Setup program from the CD. The Gaze Tracker program is installed in the C:\ERT\bin directory. It may be convenient to create a shortcut on the desktop to run the Gaze Tracker application program Synchronizing Eye Tracker with Gaze Tracker Start the Eye Tracker program on the Interface PC and download files to the Controller box. The green OnLine light (upper left of the screen) will come on, signalling that the download is complete. If you have not already calibrated the ET program, you should do that now (Chapter 5). Start the Gaze Tracker Program. The first box asks you to choose Image, Application, or Video (Figure 10-1). Since you are presenting slides individually to the subject, click on Image.

103 Figure Gaze Tracker Opening Screen. The next box asks you to "Choose an analysis mode" (Figure 10-2). Click on Load a Slide Show and then click OK. Figure Choose an Analysis Mode. A new box will open, showing a list of available slide show configurations. Highlight the demo program, which should be at the top of the list and click Load (Figure 10-3). The GT main screen will appear (Figure 10-4).

104 Figure Load a Slide Show. Figure Gaze Tracker Main Screen. When the GT main screen (Figure 10-4) appears, it will usually be superimposed on a blank screen. It may be convenient in some situations to click on the Toggle Slide View button (Figure 10-5). The GT main screen will then appear against the background of a normal desktop, including the toolbar, making other programs accessible to the user.

105 Figure Toggle Slide View. On the GT menu bar, open the ASL pull-down menu. The ASL menu presents the following options: Port Settings Transform System Type Status Indicator Test Connection Check the following: a) System Type: specify which ASL system you use. Be sure it is set to b) Port Settings: the communications port settings should be set for Com1. The baud rate should be set for 57,600 baud (not 115,200). c) The GT software asks you to "enable Demand mode data export" in the ET software. To do this you have to find the configuration file, e5000.cfg, on the Interface PC. It will be in the C:\Program Files\ASL Eye Tracker\EyeTracking\> directory, the same directory as the e5win.exe program. You can edit it using any text editor, such as Notepad, Wordpad, or the DOS editor. In the config file you should see a line: serial_data_output_format_type= If this is set to zero, reset it to 1, which means Demand mode serial output is enabled. (Zero means disabled). Save the config file with the changes. Unfortunately, when you first load the ET program from the CD, this value will be set to zero. Fortunately, once you fix the config file, it saves the settings on the Interface Computer hard drive so they don t have to be reset each time you load the program. On the GT/ ASL pull-down menu, click on Transform. The "Data Transform" box will appear (Figure 10-6). Click on the Autodetect button near the top of the box.

106 Figure Data Transform Window. Instructions will appear covering the entire screen. The screen shows a red cross covered by a blue square, positioned in the upper left of the screen. The cross/square also appears on the b/w Display screen. The instructions say: "Enable the Set Target Points mode under the Calibrate menu on the ASL Computer. Move the mouse on the ASL computer over the scene POG view. On the Interface Computer, click Calibrate on the top of the ET main screen. In the Calibrate pull-down menu, click on Set Target Points and move the Set Target Points box to one side so the Point of Gaze (POG) window is uncovered. Move the mouse pointer so it falls inside the POG window. The calibration target you used when you calibrated the ET Target Points will still be showing in the ET POG window. Notice that the red cross/blue square also shows up on the black and white (b/w) Display monitor. When you move the ET mouse into the POG window, cross-hairs will appear in the b/w Display monitor. You can move these cross-hairs around using the ET mouse. The instructions on the GT screen continue:

107 "Press the left mouse button on the GT computer when the ASL system is sending data, as shown in the scene monitor." The cross hairs on the Display (b/w) Monitor respond to mouse movement on the ET Interface Computer, as long as the ET mouse pointer is inside the POG window. Move the cross hairs in the Display Monitor and center them over the Red Cross/Blue Square as accurately as you can. The cross hairs can be made to overlap the red cross so the cross-hair lines disappear. Notice that the ET Set Target Points box gives a readout at the top of the box of the X and Y coordinates of the cross hairs as you move them (Scene X: Y: ). When the cross hairs are centered over the box on the b/w Display screen, press the left mouse button on the GT Display Computer. GT responds by moving the crosshairs/square to the lower right side of the screen. Repeat the calibration step for the target on the lower right. Move the ET mouse to line up the cross hairs, and left-press the GT mouse when the cross hairs are aligned. GT will announce that the two locations have been synchronized. Click OK to close the Notice box, then click OK to close the GT Data Transform Box Testing the accuracy of the ET to GT data transform On the GT main page, select the ASL pull-down menu. Click on "Test Connection." The Test Connection box will appear (Figure 10-7) Click on the box for "Draw X at projected position on screen." Click Start.

108 Figure Test Connection. GT sends out tests in rapid fire and counts them on-screen. The cross hairs and the Test X on the Display (b/w) Monitor should overlap. Use the ET mouse on the Interface Computer to move the cross hairs. Select a landmark in the b/w monitor screen. GT should draw a red X in the corresponding location on the GT screen. It may take a second or two for the X to settle in the GT screen. Move the cross hairs on the b/w Display monitor and be sure the test X tracks them in the subject Display monitor. The red X generated by GT should not bounce around or move erratically. When you re satisfied, click Exit on the GT Test Connection screen. In the ET/ Calibrate/ Set Target Points window, at the top of the window it shows the X- Y coordinates for the location of the cross hairs in the b/w Display monitor. E.g., Scene X: 73 Y: 123. On the GT, ASL, Test Connection window, fourth line down, it shows Orig Point (x,y). These numbers should match those shown in the ET window. The numbers are eye tracker units (ETU s), which are specific to the ASL software. ASL measures the screen as X = 260, Y

109 = 240 (Section 7.4.1). When calibration is complete, ETU s are proportional to the gaze fixation point on the screen. If the red X generated by GT doesn t track the cross hairs, you must go back through the procedure and check each step carefully. It s best if you turn everything off and start over from power-on. (The POG window will still show the calibration slide you used to calibrate Eye Tracker. The calibration slide doesn t go away unless you re-calibrate ET using Set Target Points). For example, if the baud rate is wrong, it will throw off the communication between ET and GT, the red X won t line up with the cross hairs or the landmark on the screen, and when you move the cross hairs, the red X will bounce around erratically Fine tuning the ET/GT calibration Assuming all the steps above have been completed successfully, if you run GT/ ASL/ Test Connection without moving the cross hairs and watch the alignment on the b/w display monitor, you may see that the X is slightly mis-aligned relative to the cross hairs. When you make the adjustment described here, you re going to move the cross hairs. Decide whether the cross hairs need to be moved left or right; up or down. Stop the Test Connection program and go into ASL/ Transform (Figure 10-5). At the top of the box you ll see Tracking Area with entries in ETU s. Below that there is a box for CRT Dimensions with settings for X Offset and Y Offset (to the right are boxes for Width and Height which should read 260 width and 240 height don t change them). If you change the X Offset in the negative direction, it moves the cross hairs to the left, relative to the red X. If you change the X Offset in the positive direction, it moves the cross hairs to the right. If you change the Y Offset in the negative direction, it moves the cross hairs up, relative to the red X, and if you change the Y Offset in the positive direction, it moves the cross hairs down. To enter a negative offset, you must enter an integer number in these boxes as the initial keystroke. You cannot enter a minus sign first. To enter a negative offset, enter the numeric value, then back up the cursor and insert the minus ( - ) sign before clicking OK. You have to click OK to accept the changes you ve made before you can see how they will affect the b/w display. Go back into Test Connection, and while the cross hairs are not moving, examine the position of the red X in the b/w monitor relative to the position of the cross hairs. You ll notice that changing the offset by one causes a noticeable change in alignment of the cross hairs. You have to go back and forth between Transform and Test Connection in the ASL menu until you ve got the alignment as close as you can get it, then click OK to accept the adjustments. Technical note. ASL measures the screen in Eye Tracker Units (ETU s). There are 261 ETU s on the X-axis (0 on the left; 260 on the right) and 241 ETU s on the Y-axis (0 at the top of the screen, 240 at the bottom). Gaze Tracker measures the screen in pixels. If you re using a.bmp file, there are 1024 pixels on the X-axis and 768 pixels on the Y-axis. There are (about 4) pixels per ETU on the X-axis and (about 3.2) pixels per ETU on the Y-axis. The offsets shown in ASL/ Transform/ CRT Dimensions are ETU s, not pixels. The finest-grain adjustment you can make to the offsets is one ETU. Assuming the final calibration is done carefully, the error that may be introduced at this point is ± 0.5 ETU Sanity checks

110 When you first power up for the day, ET will download files to the Controller, including calibration information from the last calibration. Running Check Target Points on the ET Calibrate menu to check for drift in the system doesn t actually check the calibration, it merely repeats what s stored in the calibration file. To check the target points, you have to run ET/ Calibrate/ Set Target Points over again. See sections and With the ET mouse in the POG window, run the GT/ ASL/ Test Connection program. Move the ET mouse around in the POG window and be sure the test X in b/w Display monitor is tracking the cross hairs and the alignment is still correct. Most of the time you shouldn t have to re-do these two calibration steps, but as components age the settings may drift. Checking occasionally makes sure the data will be recorded accurately.

111 Chapter 11: Creating a Slide Show GazeTracker (GT) will load images (slides) with the file extensions.bmp,.jpg or.gif. GazeTracker identifies a group of still images as a Slide Show. The user creates a Slide Show using the Slide Show Wizard. A Slide Show can be customized in a variety of ways, as detailed below. GT supplies a black screen and a white screen, either of which may be placed between two images, to allow the gaze position and pupil diameter to neutralize so that the effects of one slide are not carried over into the next one. Black and white screens also allow collection of pupil diameter data while the pupil is relaxed and open in response to the black screen, or contracted in response to the white screen. Of course, user-supplied slides can be inserted between stimulus slides. GazeTracker can be programmed to record external events, such as a mouse click or keyboard stroke. These events are inserted into the data stream and can be shown when a session is played back. When GT starts, the user is presented with an initial screen (Figure 10-1). To create a slide show, click on Image. The next screen asks the user to chose between loading an existing slide show or creating a new one (Figure 10-2). Checking Create a new Slide Show and clicking OK will open the Slide Show Wizard. Click Next at the first screen. Step one of the Wizard appears (Figure 11-1). Clicking the Add Image box will open a Browse window. Slides or other.bmd/.jpg files can be selected from any sub-directory that will be available to the program at runtime. The selection process is repeated until all the slides have been selected (Figure 11-2).

112 Figure Slide Show Wizard Step One

113 Figure Adding Slides to a Slide Show In Figure 11-2, nine slides have been included. After each slide is added to the list the user is asked to specify a number of parameters which will be used to control the display of this slide (Figure 11-3). The user may specify whether or not to precede this slide by a white or black screen, and if so for how long, and the time delay, if any, between display of the last slide and this one. The method of advancing to the next slide, either by timing the exposure or by keypress, (or both) must be specified. At the bottom of the screen, the user can specify whether or not, during the run, the slide preceding this one can be recalled by pressing ESC on the keyboard. Some other display characteristics, including selection of randomized presentation, can be set in the last panel at the bottom of this screen Calibration Slide The first slide in the set, NewCal is a calibration slide. The calibration time for each subject will vary, so this slide is set to remain on the screen until a mouse click is detected

114 (Figure 11-4). Figure Setting Slide Presentation Parameters The nine-point calibration slide has been adjusted, using a graphics program such as Corel PhotoPaint, so the average brightness approximates the average brightness of the stimulus slides. Using this slide to calibrate the subject minimizes the risk that the pupil and CR signals will be lost during an experimental run.

115 Figure Advance Slide when Mouse Click is Detected. The next slide in the series, Siberia.jpg, is a stimulus slide. It is set to be shown for 10 seconds (Figure 11-5). There is no keypress available which will advance to the next slide.

116 Figure Advance Slide after 10 seconds 11.2 Interspersed Text Slides The next slide is a text slide, textslide1.bmp, which instructs the subject to make ratings of the preceding stimulus slide. In this case, advancing to the next slide will be controlled by the experimenter. Only a keypress (mouse click or space bar) can be used to advance to the next slide (Figure 11-6). This is an important feature. When the subject s gaze returns to the screen after making the ratings, it takes some time for the system to re-acquire a stable pupil and CR reading. Measurement of the next slide will be inaccurate if the next timed slide appears on the screen before the pupil and CR acquisition have stabilized in the Eye Tracker. Textslide1" appears between each of the stimulus slides in the list.

117 Figure Mouse click or Space bar will Advance to the Next Slide The last slide in the series, textslide2.bmp, indicates that the experiment is over and thanks the subject for their participation. Once all of the slides have been selected, click Finish to complete the process (Figure 11-7).

118 Figure Completed Slide Show 11.3 Saving the Configuration Once the configuration (slide show) is complete, it can be saved from the File / Save Configuration menu. The Save box will appear (Figure 11-8). Note that if the user does not change the filename - Demo in this figure - when Save is pressed the old configuration, including any data it contains, will be overwritten!

119 Figure Saving the Slide Show 11.4 Editing the Configuration Once a configuration has been loaded into the program (Figure 10-2), or created and saved, the user can edit it by selecting the File menu from the main screen (Figure 10-4) and selecting the Configuration Wizard. The Configuration Wizard will show a panel of filenames, similar to that shown in Figure Slides can be deleted or added, and the display parameters can be changed. Pressing Back produces a Notice Window (Figure 11-9).

120 Figure Configuration Notice Selecting Yes in this window will take the user back to a general options window (Figure 11-3). Normally this window will be used to select global properties that will apply to all of the slides, such as stretching the image to fit the display screen, preserving the aspect ratio, and whether or not to permit a return to the previous slide using ESC. Pressing Next in the general options window advances to the first slide in the set (Figures 11-4, 11-5, 11-6), where additional parameters can be set or removed. At any step, the user can elect Back/ Next/ Apply to All/ or Cancel. When all of the slides in the set have been reviewed, the Summary will be displayed (Figure 11-7). Click Finish to exit. If the user attempts to exit the program without saving, or attempts to load or edit a different configuration without saving the first one, a warning message will appear (Figure 11-10).

121 Figure Warning Message By default, the data is saved in a file named BmpBase.mdb, in the C:\ERT\bin\> directory. This is a Microsoft DataBase file which can be edited through Gaze Tracker. The contents can be examined using Excel, but most of the file won t be meaningful to the average user. Each time a slide show is presented to a subject and the subject s data is saved, the subject s name is added to the list appearing beneath the name of the configuration. See the list of names under Exp1" in Figure Creating a New.mdb File It is desirable to create a new name for the.mdb file, either at the start of a new experiment, or to prevent an existing.mdb file from becoming too large. This is accomplished from the File menu (from the main screen, Figure 10-4) and selecting Manage Database, which will open the Manage Database Window (Figure 11-11).

122 Figure The Manage Database Window The Manage Database portion of the program can be used to select a database to load (Browse), create a new.mdb file, create a backup copy of the database, delete the database, or review it. If Edit is selected, the program assumes the user wants to edit a single record and presents a box asking for the subject s ID number (Figure 11-12). If a database file becomes too large, it may be desirable to Split it into two smaller files, each with its own filename.

123 Figure Editing a Single Subject

124 Chapter 12: Recording and Saving Data When Gaze Tracker opens, the user selects Image (Figure 10-1) and Load a Slide Show (Figure 10-2) from the opening screens. To record data for a new subject, select a previously created slide show from the list (Figure 10-3). In Figure 10-3 Exp1" has been selected. Once the slide configuration has been loaded, the GT main screen will appear (Figure 10-4). The Record button on the toolbar will be red. No slide will be in the display. The first slide will be a blank screen. If the next slide is a calibration slide (Section 11.1), click on the right arrow to Advance to the Next Slide (Figure 12-1). The Calibration slide will appear. This slide may be used to complete the subject calibration (Chapter 6) Recording When the subject calibration is complete, click the red Record button to begin recording, then click the right arrow to advance to the first stimulus slide. Figure Advance to the Next Slide If the slide show has been configured as described in Chapter 11, each stimulus slide will be on-screen for a fixed number of seconds, and will be followed automatically by a text slide. The text slide asks the subject to make ratings of the stimulus slide. Advancing to the next stimulus slide is controlled by the experimenter. During the subject s run, the Eye Tracker Point of Gaze (POG) window will show the fixation of the gaze, but it will not show the stimulus slide. The stimulus slide is shown in the b/w Scene Monitor, with the point of gaze cross-hairs superimposed. The gaze track should make sense as the subject views the slide. Pupil and CR acquisition are shown in the b/w Eye Monitor and are indicated on the ET main screen by the green Pupil and CR lights just below the Eye window. The track of the gaze and the stability of the pupil and CR signals must be monitored throughout the run. If the Pupil or CR are lost, threshold adjustments can be made on the fly. During the run the GT main screen will disappear and the entire screen will be occupied by the stimulus slide. If it becomes necessary to break out of the recording, press F2 or ALT- TAB to restore the GT main screen. From there the Stop button may be pressed to interrupt the recording Saving

125 When the last slide has been shown, the program will advance to a Save Data panel (Figure 12-2). Figure Save Data Panel If the user clicks Yes, the next panel (Figure 12-3) asks for the subject s identification number. Figure Assigning Subject ID Number

126 When OK is clicked, the next panel asks for additional identifying information about the subject (Figure 12-4). Figure Subject Identifying Information If the user attempts to exit the program, load a different configuration, or load data from another subject without first saving the data, a box appears prompting the user (Figure 12-5). If the data is not saved at this point, it will be lost Checking the Saved Data Figure Save Data before Exit Prompt Assuming the proper slide configuration and subject data is still loaded into the program, the user can check to be sure the data has been saved by opening the Analysis menu from the GT main screen and selecting Data/ View. If one of the stimulus slides is highlighted (Figure

127 12-6) the left panel will show the time the slide was exposed (15 seconds in this case) and the number of samples per second. If no data was recorded, these numbers will be zero. Figure Checking Saved Data Clicking on the + box next to Gazepoints will show all the X-Y sample points that have been recorded (Figure 12-7). The pupil diameter isn t show in this panel, but it is included when the data is exported (Chapter 14).

128 Figure Recorded Sample Points for One Slide While the subject s data is still in the program, it can be checked by clicking on the Play button (Figure 12-1), which will be green. The gaze path for a particular slide will be replayed. This insures the integrity of the data, but does not guarantee that it has been saved.

129 Chapter 13: Creating Look Zones Gaze Tracker refers to Areas of Interest in a slide as Look Zones (LZ). Look Zones can be created either before or after data has been recorded. They can be analyzed in terms of: 1) the amount of time the fixation point was within the LZ during the exposure; 2) the number of times the LZ was entered during the exposure; 3) the number of fixations (as defined by the GT program) which fell within the LZ. (Criteria for Fixation points are defined in the Analysis/ Gaze Trail Options box. See Figure 14-1.) LookZones can be any size or shape and can overlap in whole or in part. Before creating Look Zones, it s a good idea to make a back-up copy of the.mdb file where the configuration and subject data is stored. You can do this using the File / Manage Database window, using the Backup command (Section 11.5), or use Windows Explore, find the database file in C:\ERT\bin\> and copy it to another location or rename it. Begin by loading a previously created slide show from the list (Figure 10-3). In Figure 10-3, Exp1" has been selected (not the data for an individual subject). If there will be more than one or two obvious LZ s per slide, it is a good idea to make a printed version of each slide and make a preliminary sketch of the look zones that will be created. This simplifies the task of drawing the LZ s and reduces confusion in interpreting the data, especially if the LZ s are given cryptic names like L1, C3, etc Creating Look Zones Once a configuration (slide show) has been loaded, the GT main screen will appear (Figure 10-4). To create a Look Zone, click on the LookZones button on the main screen and select Apply. Use the Right/Left arrows to advance to the slide you want to work on (Figure 13-1). Figure Apply Look Zones Box The Apply Look Zones box will appear overlapping the middle of the slide. Use the mouse to move the box to one side while creating the LZ s. To create a rectangular area, move the cursor to one corner of the area, click and hold down the left mouse button, and drag the cursor to the opposite corner. Releasing the mouse button closes the area. Note that if a larger zone completely encloses a smaller one, the smaller one must be created first, to prevent its being hidden by the larger one.

130 To create a non-rectangular area, move the cursor to any point along the contour and right click to create a point. As the cursor is moved, a straight line will extend outward until another right click is made to anchor the line. Make successive right-clicks around the perimeter to create the LZ. At the final point, left-click the mouse to close the shape. 13-2). Once an LZ has been closed, the Look Zone Properities dialog box will appear (Figure Figure Look Zone Properties Dialog Box Use this box to enter a name for the LZ. If a group of LZ s is to be analyzed together, the owner (group) can be specified, or a new group can be created. Close the box by clicking OK. When all the LZ s for a slide have been created, go back to the Apply Look Zones box (Figure 13-1) and click Stop.

131 13.2 Editing and Manipulating Look Zones An LZ can be resized by left-clicking on any corner or vertex and dragging it to a new position. The entire LZ can be moved by left-clicking on any point inside the LZ and dragging the zone to a new position. An LZ can be deleted by right-clicking on any point within the LZ and selecting Delete from the Look Zone pop-up menu. Correcting an LZ containing mistaken vertices may require deleting it and starting over. Selecting Options from the Look Zone pop-up menu allows the user to change the Look Zone Properties (Figure 13-2). Selecting View Data from the Look Zone pop-up menu opens the Recorded Data dialog box (Figure 12-6). The highlight will be on the current LZ. Current LZ information appears on the left side of the box. Clicking the + beside the name of the LZ shows the information currently available for this LZ. E.g., if external events were recorded while the gaze was inside the LZ, it will be shown here (Figure 13-3). Figure Look Zone Data Dialog Box If an LZ is altered, checking Update Gaze Tracker Data Window as selection changes in the Data Dialog box will force the program to re-allocate the X-Y sample points and fixation points to reflect the altered shape of the LZ. In turn, the statistics for the LZ will be updated Saving Look Zones LZ s may be saved using the File/ Save Configuration menu (Figure 13-4) to re-save the configuration.

132 Figure Save Look Zone Data Unless the name of the configuration is changed, the user must answer NO in the dialog box which follows after Save is clicked. Otherwise the data saved under that name ( Exp1" in this case) will be overwritten! Answering No causes the changes to be saved without overwriting the data. The next time the configuration is loaded, the LZ s will be available. From the GT main screen, use the LookZone menu Show/ Hide selections to make them visible. For example, it may be desirable to Show the LZ s in a slide when data is being played back.

133 Chapter 14: Data Analysis Using Gaze Tracker There are two ways to export data from the Gaze Tracker main screen (Figure 10-4): Analysis/ Data/ Export, or Analysis/ Multiple Subject. In either case, the data may be exported as an ASCII (text) file or as an Excel (.xls) file. For 3-D displays, etc., the use of the Gaze Tracker Graphing Wizard is explained in the file: GTManual.pdf in the C:\ERT\Manuals> directory on the Display computer. To export data, the user must first load the file containing the configuration and the data (Figure 10-3). In Figure 10-3, the configuration for Exp1" is selected. If Analysis/ Data/ Export is chosen, only the data for the subject currently loaded into the program can be output. The subject s data can be loaded at startup, or by using the File/ Load Data routine from the GT main screen. There is no provision in this section for outputting data for more than one subject at a time Analysis/ Data/ Export: Outputting Raw Data and Summary Statistics The first method outputs all of the X-Y sample points, the associated pupil diameter in X and Y dimensions for each sample, and the time (starting from zero when Recording was begun) at which the sample was taken. If fixations are detected, they are listed below the raw data. If Statistics are requested, a table is appended showing summary data. If Look Zones have been attached to the slide, this information is included in every row of the data. The output is generated by selecting Analysis/ Data/ Export from the main screen. (If Analysis/ Data/ View is selected, then Export can be selected from the View window.) The following screens will be displayed: Choose the data export destination: Excel Text File Choose the source of data to export Slide data (default will be checked) Look Zone data (which is checked in addition to Slide Data ) Choose the slide(s) to export (Check boxes to the left of each slide name) Choose data type to export Eye position data Pupil data Input Event Data (for analyzing key-press or mouse-clicks inserted into the data stream) Specify eye position and pupil data export options

134 Export gaze data Export fixation data... Export statistics Specify time filtering options Export data for the entire data slide Specify time interval to export Summary - shows selections made above. Perform After Perform is clicked, a Save box will open giving the usual options: the directory to which the file is to be saved, the filename, and the file extension. The program default for text output files is.out Text File Output: X-Y Sample Data The following table gives an example of a text file output. Only one slide, called couple laughing.jpg is analyzed. The first part of the table is a heading: Slide, Number, x, y, Pupil X, Pupil Y, Time, LZ Name Slide, Number, x, y, Pupil X, Pupil Y, Start Time, End Time, Duration, LZ Name Two headings are given at the top of the table. Only the first line of the heading applies to the X-Y sample data. The next part of the table gives the X and Y coordinates for the gaze, the (uncalibrated) pupil diameter in X and Y dimensions, the time at which the sample was taken, starting at zero, and the name of the Look Zone or Zones, if any, into which the gaze fell. In this example, the slide was exposed for 15 seconds, so there is a long list of sample data. The data is commadelimited: Sample Pupil Pupil Name of slide Number X Y X diameter Y diameter Time LZ C:\Photos\couple laughing.jpg, 1, 510, 506, , , 0.000, R1 C:\Photos\couple laughing.jpg, 2, 510, 506, , , 0.015, R1 C:\Photos\couple laughing.jpg, 3, 510, 506, , , 0.031, R1 C:\Photos\couple laughing.jpg, 4, 510, 506, , , 0.047, R1 Samples 5 through 897 are deleted. C:\Photos\couple laughing.jpg, 898, 633, 428, , , , C:\Photos\couple laughing.jpg, 899, 676, 443, , , , During the last two samples, the gaze was not directed to any of the Look Zones defined for this slide, so no Look Zone information is given for these two samples.

135 Text File Output: Fixation Data After the raw data, the output continues showing data for fixations. The second line of the heading applies to this part of the table. Fixations are defined in the program in the Analysis/ Gaze Trail Options box, under Fixation Options (Figure 14-1). Under the default settings, a fixation occurs when the gaze stays within the same area on the screen, not more than 40 pixels in diameter, for at least 0.2 sec. There must be a minimum of three valid samples within this area/time frame. These settings may be adjusted by the user under Fixation Options. Figure Choosing Defaults for Fixation Options from the Gaze Trail Options Window In the example, 31 fixations were detected during the 15 second exposure. The fixations are given consecutive numbers. The example below shows only the first three fixations. Each row in the data shows the fixation number, the mean X and Y location of the gaze, the mean diameter of the pupil in X and Y dimensions, the start time (from zero) and end time of the fixation, the duration of the fixation in seconds, and the Look Zone or Zones, if any, in which the fixation occurred. Fixation Mean Pupil Mean Pupil Start End Name of Slide Number X Y X diameter Y diameter Time Time Duration LZ

136 C:\Photos\couple laughing.jpg, 1, 388, 116, , , 0.375, 0.672, 0.297, L4, L1 C:\Photos\couple laughing.jpg, 2, 667, 108, , , 0.703, 1.578, 0.875, R2, R1 C:\Photos\couple laughing.jpg, 3, 406, 89, , , 1.609, 2.297, 0.688, L4, L1 A summary for the slide is displayed in a table of Statistics. Following the Statistics, a summary is given for each of the seven Look Zones defined for this slide. Data for only one of the Look Zones is shown in this example. The table concludes with an analysis of time spent outside of all LZ s. Overall metrics for slide C:\Photos\couple laughing.jpg: Time span shown in this file : All Total time shown (seconds): Total time tracked (seconds): Total tracking time lost (seconds): Total fixation duration (seconds): Total time nonfixated excluding gaps (seconds): Percent time tracked: Percent tracking time lost: Percent time fixated: Percent time nonfixated excluding gaps: Percent time fixated related to time tracked: Percent time nonfixated related to time tracked: Average pupil x diameter: Average pupil y diameter: Average pupil area: Pupil x diameter std dev: Pupil y diameter std dev: Pupil area std dev: Number of fixations: 31 Fixation count / Total time shown: Fixation count / Total time tracked: Average fixation duration (seconds): Std dev fixation duration (seconds): Average pupil x diameter in fixations: Average pupil y diameter in fixations: Average pupil area in fixations: Pupil x diameter std dev in fixations: Pupil y diameter std dev in fixations: Pupil area std dev in fixations: Number of gazepoints: 899 Gazepoint count / Total time shown: Gazepoint count / Total time tracked: Fixation points in zones: 30 Percent fixations in zones: Gazepoints in zones: 847 Percent gazepoints in zones: LookZone 1: C1 Number of times zone observed: 5 Number of fixations before first arrival: 12 Duration before first fixation arrival (seconds): Total time in zone (seconds): Percentage of total fixations before first arrival:

137 Percentage of total slide time before first arrival: Percent time spent in zone: Average pupil x diameter: Average pupil y diameter: Average pupil area: Pupil x diameter std dev: Pupil y diameter std dev: Pupil area std dev: Fixation count: 2 Percentage of total fixations: Total fixation duration (seconds): Total time not fixated (seconds): Percent time fixated related to time in zone: Percent time nonfixated: Percent time fixated related to total fixation duration: Fixation count / Total time in zone: Fixation count / Total fixation duration in zone: Average fixation duration (seconds): Std dev fixation duration (seconds): Average pupil x diameter in fixations: Average pupil y diameter in fixations: Average pupil area in fixations: Pupil x diameter std dev in fixations: Pupil y diameter std dev in fixations: Pupil area std dev in fixations: Gazepoint count: 62 Gazepoint count / Total time in zone: Gazepoint count / Total fixation duration in zone: LookZone 3: Outside of all LookZones Number of times zone observed: 0 Number of fixations before first arrival: 30 Duration before first fixation arrival (seconds): Total time in zone (seconds): Percentage of total fixations before first arrival: Percentage of total slide time before first arrival: Percent time spent in zone: Average pupil x diameter: Average pupil y diameter: Average pupil area: Pupil x diameter std dev : Pupil y diameter std dev: Pupil area std dev: Fixation count: 1 Percentage of total fixations: Total fixation duration (seconds): Total time not fixated (seconds): Percent time fixated related to time in zone: Percent time nonfixated: Percent time fixated related to total fixation duration: Fixation count / Total time in zone: 0.000

138 Fixation count / Total fixation duration in zone: Average fixation duration (seconds): Std dev fixation duration (seconds): Average pupil x diameter in fixations: Average pupil y diameter in fixations: Average pupil area in fixations: Pupil x diameter std dev in fixations: Pupil y diameter std dev in fixations: Pupil area std dev in fixations: Gazepoint count: 52 Gazepoint count / Total time in zone: Gazepoint count / Total fixation duration in zone: If other slides were selected for analysis, these are shown next Outputting a Summary for Multiple Subjects It is often desirable to ignore the raw data in favor of a summary of data for the Look Zones. In this example, an Excel file is created. The output is generated from the main screen from Analysis/ Data/ Multiple Subjects. The first box asks for a definition of the type of output (Figure 14-2). An Excel data sheet with fixation data has been requested (Time, Entrance, and Fixation are mutually exclusive). Figure Selection of Data Type

139 The next box allows the user to specify the type of data to be output for each Look Zone (Figure 14-3). The total number of fixations per LZ has been requested. Figure Summary Data for Look Zones

140 The next box allows the user to create summary tables, summarizing across slides, or across subjects (Figure 14-4). In this case, no summary is requested. Figure Summary Data Options The program will use the current data base by default, but the next slide allows the user to select a different data base, if desired. The user selects the slides to be included in the analysis in the next window (Figure 14-5). A thumbnail of the slide is shown on the left as each slide is selected. If Look Zones have been attached to the slide, this information is also shown. In this case, only two slides have been selected. The repeating text slide gives instructions to the subject and is not analyzed.

141 Figure Slide Selection If Look Zones have been attached to the slides selected, they can be included for analysis (or not) in the next box (Figure 14-6).

142 Figure Selecting Look Zones The subjects to be included in the analysis are selected in the next box (Figure 14-7). Unfortunately, using the version of GT we have in the lab, ver FULL ASL, the program hangs if more than one subject is selected for inclusion in the analysis.

143 Figure Subject Selection After the subjects have been selected, a summary screen appears (Figure 14-8). If the name of a slide is highlighted, the associated Look Zones to be included in the analysis are shown in the third panel.

144 Figure Summary When Finish is selected, Excel opens with the data. If the user wants to save the data, it must be saved before exiting Excel. Figure 14-9 shows a truncated example of an Excel data file.

145 Figure Three Factor Excel Data File 14.3 Working Around Program Limitations To prevent the program (ver ) from hanging, the Excel file shown in Figure 14-9 must be created by outputting the data for one subject at a time. Each subject becomes a separate Excel file. The data for each successive subject must be copied to the Clipboard and Pasted into the Excel file for the first subject, until all the data has been assembled. This is an extremely tedious procedure, especially since several dependent variables may be analyzed. For example, the user may wish to examine the total time the gaze was within each LZ, the total number of times the LZ was entered during the exposure, or the number of fixations occurring within the LZ, regardless of when they occurred (Figure 14-2). It may also be necessary to examine the sequence of entries/exits from the LZ s, or it may be necessary to select a portion of the data for analysis. For example, the first 2 or 3 seconds after an exposure is begun are critical times when the subject forms his/her initial impressions of the slide. From this first impression later strategies may develop. If there are 60 subjects and the slides are analyzed for even four of the several possible dependent measures, then 60 x 4 = 240 separate Excel files which have to be created and cut-and-pasted. It is unfortunate that Eye Response Technologies has not been able to remedy this problem. An updated, Demo version of GT is available (ver DEMO) that does not hang, but it still only outputs data for one subject at a time. The latest version of GT, version 5.x, has also failed to correct this problem. If more than one subject is selected, instead of a three-factor output (Figure 14-9), the program outputs a two-factor Excel file (Figure 14-10), with Subjects as one factor and Slides as the other. The output ignores LZ s within Slides. In addition, version 5.x will not read an.mdb data file created with earlier versions of GT.

146 Average Time per LookZone Subjects Slide 1 Slide 2 Slide 3 ( secs) ( secs) ( secs) Peanut Nate Allen Michaels Slide Legend Slide 1. C:\New Photos\dadsonsoccer.bmp Slide 2. C:\Photos\brushing.jpg Slide 3. C:\Photos\moda_cookie.jpg Figure Two Factor Excel Data File

147 Appendix One Startup Instructions A1.1 Short Startup Checklist Turn on both computers. Start both programs. EyeTracker: Start Upload (green light comes on) EyeTracker: nine cal. circles appear in POG window if not, ET: Calibrate / Set Target Points EyeTracker: Pan/Tilt Tracking set to Manual Seat subject, adjust chair, mount MHT sensor EyeTracker: Pan/Tilt / Setup four slide switches all to right (max) MHT head rotation limits not checked EyeTracker: Illuminator ON, Illumination slider ½ to right Focus camera on subject's eye; camera autofocus Off. EyeTracker: raise Pupil slider until pupil is acquired, then CR slider and acquire CR. Be sure both are stable. EyeTracker: MHT Menu / Enable If the camera is focused on the eye, click EyeTracker: * button to engage MHT as long as the camera doesn't fly away from the eye and lose the image, you're OK. EyeTracker: Pan/Tilt / Setup box, Use MHT rotation limits 9-point cal screen on subject's display (use XnView, full-screen view) EyeTracker: Calibrate / Eye Calibration Store data for points 1-9 Check calibration: Have subject look at each point & check accuracy (look in lower b/w monitor, switched to show same thing subject sees; should have cross-hairs superimposed to show gaze target); fix any points that are bad (using Calibrate / Eye Calibrate) GazeTracker: start recording; minimize GT panel; F2 to restore. Watch for flashing GT panel & minimize if it comes back. Finished recording: EyeTracker: Pan/Tilt Tracking set to Manual Remove headband

148 GazeTracker: check data was recorded, either playback data or Analysis / View / Data and check X/Y contents of data

149 A1.2 StartUp Checklist A1.2.1 For This Session Turn on both black power boxes (monitors will come on; you will hear fans running). Everything should come on, including: Controller (box underneath the b/w monitors, red light inside on/off switch); Scan Converter (AverKey Micro = small grey box on floor - this box has no on/off switch); Camera (if camera doesn't come on, switch is on back. Camera will rotate in self-test when power comes on). MHT control unit (on shelf next to subject's feet = "Flock of Birds" set to "FLY" on back of box - this box has no on/off switch). Turn on both computers. Sign on to Display computer as Observer. Password = Tracker$ Password is case-sensitive. Sign on to Interface computer as Observer. There is no password, just press Enter. Start Eye Tracker (ET) program (icon for Eye Tracker) on the interface computer (white Dell) and start Gaze Tracker (GT) program on the display computer (black Dell). On GT, you have to load a "config" file to get to the main screen - doesn't matter which one, you can change later. ET will offer to upload software to Controller. Click on Start Upload. Magnetic Head Tracker (MHT) has been calibrated previously. If transmitter and camera have not moved, this does not need to be repeated. For MHT Calibration, see Manual, Chapter 5. ET and GT programs have been synchronized. Does not need to be repeated if monitor has not been changed,. Repeat occasionally to check for system drift. For synchronizing the two programs, see Manual, Chapter 10. The file "GTSetup.doc" on CDROM disk also contains instructions. Manuals and notes are also on Display Computer in: c:\ert\bin\manual\> When Upload to ET Controller is finished, green "Online" light will come on (top left of ET main screen) and camera will move. If Upload fails, the Online light stays red. Try closing Eye Tracker (X on top right corner of screen), turn off the Controller, wait 20 seconds, then turn on Controller and start ET again. The downloaded information stays in the Controller as long as it's not turned off. In re-starting, ET may "grab" it. Turning off the Controller makes sure it forgets any garbled information. If Upload still fails, check to be sure everything is on - all the boxes on the floor under the table - check all the connections, turn Controller off for several seconds and run Configure / Upload to Control Unit again. Follow instructions/hints in Upload box if it fails a second time. Note lower right of ET main screen, below "Scene POG" window, is blank (MHT tracking is not enabled). ET / Configure Menu / System Settings box should show (this should not have changed since last session):

150 System Type: Pan/Tilt Optics with MHT Option Eye Camera Speed = 60 Number of Eye Position Fields to Average = 4 Video Scene Source = AutoSelect (or Remote Scene) Interface Port = Com1 Baud Rate = 115,200 Don't change anything else - click OK Continuing to check settings: After Upload is finished you can test to be sure ET and GT are talking to each other: In ET / Calibrate / Set Target Points drag the ET cursor so the + is showing in the POG window. In GT / ASL / Test Connection Click on "Draw X at projected position on the screen" Click on "Start" Moving the ET cursor in the POG window should cause the red X in GT window to move around synchronously. In GT / Test Connection window Click Stop. Click Exit. In ET / Set Target Points screen Click on "Quit without Saving" to close Set Target Points box. If this fails, you have to make sure GT and ET can talk to each other. See synchronizing ET and GT, above, and Manual Chapter 10. ET Set Target Points has been done previously. This should be repeated occasionally to correct for system drift, but need not be repeated each time. See Manual, Chapter 5. Also, ET Manual, Section 5.1 on pages 36-38, and the file "GTSetup.Doc" on CDROM disk. (Manuals and notes are also on Display Computer in: c:\ert\bin\manual\>) Nine white circles should appear in ET POG window, indicating that the Target Points have been set and config. has been saved. If not, you have to repeat Set Target Points: On the Display Computer, start XnView from the desktop icon. Load "NewCal.bmp" file in C:\Photos> or in XnView default folder. Convert so the image fills the screen: View / Full Screen.

151 In ET / Calibrate / Set Target Points Turn on b/w monitor showing long-range lens. Set the A/B switch on the monitor so the 9-point target pattern shows in the monitor. Note the black crosshairs. Moving the ET mouse will move the black crosshairs. Put the crosshairs over Spot 1 as precisely as you can and press Enter on the Interface computer keyboard. The number in the Set Target Points box will increment. Repeat for points 2-9. Click "Save Target Points and Quit" (very important) Close Set Target Points box The white circles, indicating 9 target points, will remain showing in POG window. Switch A/B switch on lower b/w monitor back to long-range lens. Hit ESC on display computer keyboard to restore XnView panel, then File / Quit to exit XnView program. Then start GT. MHT System: ET / MHT Menu / Select MHT System should say Ascension Flock (click OK) (should not have changed since last session) Bottom, left side of ET main screen: Pan/Tilt Tracking set to Manual A1.2.2 For this Subject 1) Seat the subject, adjust the height of the chair and the height of the monitor if necessary. Subject's line of sight should be straight at the screen, or looking slightly down toward the center of the monitor. Camera looks up at subject's face from below. 2) Mount the magnetic sensor above the eye to be tracked. The cord from the sensor goes over the subject's right shoulder, even if the left eye is being tracked. The arrow on the sensor points up. 3) Pan/Tilt Menu - Setup -> Pan Tilt Setup Box Slide switches for Shutter, Iris, Gain, and Zoom all set to right (Max). "Use MHT Head Rotation Limits" not checked. (May still be set from last time.) It is NOT necessary to enter a number in the box for MHT Sensor-to-eye offset. You only use this measurement if the magnetic sensor cannot be placed right over the eye, e.g., if it's over the person's ear. Doesn't change from last time. Should be set = zero. Click "Set" and close Pan/Tilt Setup box (click X on top right to close box). 4) Turn Illuminator ON. (Check box on top left of screen, under "Power") (Checking the "Camera" box in software doesn't do anything. Camera must be turned on with switch on back of camera.) 5) On Main screen, under "Discrimination" there are three sliders.

152 Illumination slide should be 1/2 of the way to the right (brighter). You should see a circle of light, or lighter area, in b/w monitor when you raise illumination slider. Pupil slider all the way to the left. CR slider all the way to the left. Pan/Tilt Tracking (bottom of screen) set to Manual Camera Auto-focus OFF. 6) Aim the camera using the hand-held remote (bounce the IR beam from the control off of the mirror in front of the camera). Use the two b/w monitors, one for long-distance, the other for close-up. Center the subject's eye in the close-up monitor. Don't need good pupil or CR recognition at this stage. You do not need to set ET: Pan/Tilt Menu, Set Home. AutoFocus may be ON or OFF at start-up (power on). The only way to turn camera auto-focus on/off is with the hand-held remote control (button on upper left). The only way to tell if auto-focus is on or not is to observe the behavior of the camera. There is no indicator in software, and no indicator on the camera. To focus, try zooming "out" (marked on control) and hold rocker down. If camera movements are reversed, hit the blue "L/R Direction Set" button on the bottom of the hand-held control, then press "Std" (1) on hand held control. Checking the button for "Reverse Left/Right" or "Reverse Up/Down" in the program Pan/Tilt Setup box only affects the software, not the handheld controller. 7) Subject looking straight ahead. With illumination slide about 1/2 of the way up, increase Pupil slider using the right arrow until: Watch the b/w close-up monitor as you move the slider White circle will form and stabilize as you continue to increase the slider, indicating pupil recognition. White cross hairs will appear in the b/w close-up monitor. On Main screen, below Eye box, next to "Recognition," the green light for "Pupil" will come on. It should stay on. If pupil is being lost, this light blinks on and off. Purple-pink cross-hairs also appear in "Eye" panel on main screen. If speckling appears, or crosshairs jump to other objects, slider is too far to the right. If circle doesn't form, lower Pupil slider, raise Illumination, and try again. Make sure pupil is stable & you don't lose it. Wait a minute or two and watch as subject's eye moves around. At this stage, if eye moves slowly, and doesn't move out of range (out of frame formed by b/w close-up monitor), the software will track the pupil and keep it centered in the close-up monitor as the eye moves. 8) Subject still looking straight ahead. Move CR (corneal reflection) slider slowly to the right until black cross-hairs pop up on b/w close-up monitor. Blue crosshairs appear in "Eye" window on main screen and green "CR" light under "Recognition" will come on. Wait a minute to be sure you don't lose the CR. Pan/Tilt Tracking still on Manual. Set Camera Auto Focus ON, using remote control (red indicator light on front of camera will blink on, then off).

153 If Pan/Tilt Tracking is switched to Auto (but don't do it yet), the camera will move to track slow movements and will keep image of pupil centered in b/w close-up monitor. But rapid movement of the eye out of the frame will loose the pupil and tracking will stop. 9) On the Main screen / MHT menu / Enable - click "Enable" to establish communication between MHT and software. MHT X, Y, and Z coordinates will appear in "Transmitter" box, below Scene POG on the main screen. Offset readings will all be zero. The X-Y-Z button (*) at the top of the main screen will turn black. Press * button (three crossed axes) on top of main screen (or use Pan/Tilt Menu / MHT Sensor Calibration). When you press the * button, Pan/Tilt Tracking (bottom left of screen) will switch to Auto by itself. At this point the camera should follow eye movements and head movements (camera will lag slightly). If the camera veers away from the subject and you lose the eye image when you click * you have to start over. Auto focus should keep the eye in focus in the close-up monitor. If you move the eye out of range, camera will follow, and will re-acquire the pupil as soon as it comes back in range. Test this by rotating chair or have subject turn away, to be sure magnetic tracking is working. During period when pupil is lost, data for X-Y gaze location is set to zero, and pupil diameter = zero. Go back into Pan/Tilt Menu / Setup, and check "Use MHT Head Rotation Limits." Click Set and close Setup box. This prevents the camera from trying to follow the pupil when it is too far out of range (data set to zero outside these limits). 10) Subject calibration Bring up 9-point calibration slide on screen, using XnView program icon on the Dell. ET Calibrate Menu / Eye calibration (or use short-cut button) Move the Eye Calibration box so it's not inside the POG window. Don't click inside POG window as you do this calibration, or you may move one of the calibration points by mistake. Ask subject to look at Point 1. Watch b/w close-up monitor to be sure you don't lose the pupil or CR. If you do, readjust the three sliders under Discrimination on Main screen. It's O.K. to re-adjust these "on the fly" (anytime). Click "Store the Data for Current Point." The counter will automatically advance to point 2. Ask the subject to look at point 2, click "Store the Data for Current Point." Repeat for points 2 through 9. After "Store Data for Current Point" is clicked for point 9, window will close automatically If all else fails and you're still losing pupil/cr recognition, go back to step 3, Pan/Tilt Menu / Setup / Pan Tilt Setup Box, and change settings for Iris, Shutter speed, or Gain. See

154 Manual, Chapter 5, Figure 5-7, or Section on page 50 of the Eye Tracker manual. These adjustments work just like they do on a camera. Decreasing "iris" is the same as using a smaller f-stop on a camera. Cut back on the amount of light getting into the camera using Iris and Gain, and try again. Another alternative is to lower the room lights and/or dim the brightness on the monitor, which will make the subject's pupil open and make it easier to discriminate. 11) Check accuracy of subject calibration. You must check this. The cross-hairs during calibration (step 10) only indicate which point the subject is supposed to be looking at next. They do not accurately reflect eye position. Ask subject to look at each point (1-9) again. The position of + in POG box, and the black cross-hairs in the scene monitor (the b/w monitor with the black dot) should show crosshairs, indicating eye position, superimposed on the target points. (If b/w monitor is still showing the long-lens, flip the A/B switch on the bottom of the monitor so it shows the scene as the subject sees it.) If the calibration is not accurate, repeat calibration for any spots that are bad, using Calibrate / Eye calibration (see step 10). You don't have to do them all over again, just the ones that are "off." 12) Give the subject instructions; start recording. Minimize the GT panel during recording so it doesn't interfere with the subject display. F2 to restore GT panel when run is finished. ALT Tab to restore GT panel during record (to break out). 13) After recording is finished, before removing headband, set Pan/Tilt Tracking (bottom of main screen) to Manual. Turn off Illumination. 14) Check to be sure data was recorded GT will ask you repeatedly if you want to save the data. Whether data saved or not, GT / Playback the data. It should show the fixation points and the gaze path for each slide when you do this. You can Pause playback once you've checked one or two slides. If slide appears with no gaze-path, no data was recorded. Alternatively (assuming data was saved), Analysis / View / Data Click on + next to one of the images the subject looked at Click on + next to GazePoints The program records data points sixty times per second. You should see several seconds worth of data, with X-Y coordinates, under each image file you check. If the X-Y coordinates are identical for each sample, no data was recorded.

155 Appendix Two Menus A2.1 Eye Tracker Menus File Menu New Close Start Recording Stop Recording Mark Event Exit (Alt X) Configure Menu System Settings System Type Eye Camera Speed Number of Eye Position Fields to Average Scene Video Source Interface Port (Com 1) Baud Rate (115200) Read Configuration Save Configuration Upload to Control Unit Calibrate Eye Calibration Quick Calibration Set Target Points Check Target Points Eye Position Offset Quick Offset (P) Reset Offsets Read Calibration Save Calibration Eyehead Setup Gimbal Test Integration Data Display Initialize Enable Stationary Scene Camera Stationary Scene Camera Target Points Read Environment Save Environment As Pan/Tilt

156 Setup Home Set Home MHT Pan/Tilt Calibration MHT Sensor Calibration Read Setup Save Setup MHT Enable Set Boresight Reset MHT Select MHT System Read Head Tracker Configuration Save Head Tracker Configuration Misc View Data File (but no browse) Reset Elapsed Repaint Desktop Save Edge Data Buffer Help About (gives version number) A2.2 Eye Tracker Buttons Open Data File (but this button opens a SAVE dialog box) Close Data File Start Recording Stop Recording External Events (0 9) Stationary Scene Camera (SSC) MHT Sensor Calibration Upload Software to Control Unit Set Target Points Check Target Points Eye Calibration Data Display (DD) Gimbal Test Eye Head Integration (EHI) Enable MHT (MHT)

157 A2.3 Eyenal Menus File Open (Ctrl O) Print Setup 1 (data.eyd files) Exit once file is loaded, shows the following: New (Ctrl N) Open (Ctrl O) Close Save (Ctrl S) Save As ASCII Export (as Excel file) Delete Print (Ctrl P) Print Setup 1 filename Exit View Toolbar Status Bar Fixation Make a new Fixation file Open AOI Make new AOI file Open Edit Fseq Make a new fixation sequence file Open Dwell Make a new Dwell file Open Pupil Make a new pupil diameter analysis file Open Batch Batch Fixation Batch Fix Sequence Batch Dwell Group

158 Fix Sequence Dwell Open Window New Window Cascade Tile Arrange Icons Help Help Topics About Eyenal A2.4 Fixplot Menus (no underline for ALT F, etc.; no keyboard shortcuts) File New (Ctrl N) Open Image (Ctrl O) Print Setup Recent File Exit Frame Grabber Select Device Close Device Setup > Select Input Source Synchronous Asynchronous Start Pass Thru! Stop Pass Thru! View Toolbar Status Bar Options Preferences Output Type Help Help Topics About FixPlot Buttons F1 for Help Contents Index Back Print

159 (can copy or annotate Help files using Edit menu) Create a New Document Open Document Save Copy selection to clipboard Paste (Insert) Clipboard Contents Select a new Zoom Zoom In Zoom Out Go one step back (Ctrl Rt (Left?) arrow) Go one step Forward (Ctrl Rt Arrow) - step not steep Display program information (version number and copyright) Help for buttons, menus, and windows (click this button, then select an object on which to get help) (help files for some topics do not exist) A2.5 Gaze Tracker Menus File New Configuration Close Configuration Manage database Configuration Wizard Load Data Save Configuration Save Recorded Data Exit Operations Record Rewind Play Pause Stop Play to Video File Recording Options Remote Control Shift Data Questionnaire New Questionnaire Edit Questionnaire Load Questionnaire Save Questionnaire View Answers Look Zones Apply Look Zones Show Look Zones

160 Hide Look Zones Set Active Collection Load Look Zone Layout Save Look Zone Layout Analysis Data > View Filter Export Print Graphing > 3D Contour Look Zone Order Excel Pupil. View Toolbar ASL Port Settings Transform System Type Status Indicator Test Connection Help Contents Search Index Tutorial About Gaze Tracker A2.6 Gaze Tracker Buttons Record Rewind Play Pause Recording/Playback Stop Move Back One Slide Rewind Slide Play Slide Next Slide Toggle Slide View (Desktop/Slide) Refresh Data Window Create New Look Zones Display (View) Data Adjust Gaze Trail Options

161 Export to Text or Excel File 3D Analysis Contour Graph Show the Look Zone Order Graph Excel Graph Pupil Graph Help Table of Contents