A Turning Cabin Simulator to Reduce Simulator Sickness

Transcription

1 A Turning Cabin Simulator to Reduce Simulator Sickness Ronald R. Mourant, Zhishuai Yin Virtual Environments Laboratory, Northeastern University, 360 Huntington Ave., Boston, MA, USA ABSTRACT A long time problem associated with driving simulators is simulator sickness. A possible cause of simulator sickness is that the optical flow experienced in driving simulators is much different from that experienced in real world driving. With the potential to reduce simulator sickness, a turning cabin driving simulator, whose cabin rotates around the yaw axis was built. In the multi-projector display system, algorithms were implemented to calibrate both geometric distortions and photometric distortions via software to produce a seamless high-resolution display on a cylindrical screen. An automotive seat was mounted on an AC servo actuator at the center of the cylindrical screen. The force feedback steering wheel, and gas and brake pedals, were connected to the simulator s computer. Experiments were conducted to study the effect of optical flow patterns on simulator sickness. Results suggested that the optical flow perceived by drivers in the fixed base simulator was greater than that in the turning cabin simulator. Also, drivers reported a higher degree of simulator sickness in the fixed base simulator. The lower amount of optical flow perceived in the turning cabin simulator is believed to be a positive factor in reducing simulator sickness. Keywords: driving simulator, optical flow, simulator sickness, multi-projector display 1. INTRODUCTION Driving simulators are widely used in many public and private research facilities for various purposes. By reproducing real driving conditions, driving simulators allow researchers to study human factors in a safe manner, develop and evaluate driver assistance systems, and test functionalities of in-vehicle information devices. Existing driving simulators range widely in capability and complexity. Some examples of highly sophisticated simulators are the National Advanced Driving Simulator located in Iowa [11], and the Toyota Driving Simulator [6]. They both have an actual car on a platform inside a six-axis dome which also serves as a 360-degree video screen. Simple driving simulators can be merely a desktop PC with a game steering set. A great number of driving simulators fall somewhere between these two extremes to provide drivers adequate immersive and realistic driving experiences with affordable expenses [1][2][9]. As suggested and generally agreed by previous studies, a wide FOV display would help in enhancing the sense of presence in virtual environments. However, it s also has been reported by some researches that simulator sickness seems to occur more often in a wide FOV display [3][8]. Optical flow patterns generated in the simulator are considered to be a factor in simulator sickness. Our goals are to enhance the realism of the driving experience as well to reduce simulator sickness. This study introduces a mid-level driving simulator with a one-degree of freedom (DOF) motion platform. With the turning cabin driving simulator, experiments were carried out to study the relationship between simulator generated optical flow and simulator sickness. 2. DESCRIPTION The turning cabin driving simulator consists of the following components: a) An image display system that is composed of a 360-degree cylindrical screen and three LCD projectors. b) A 1 DOF driving simulator platform (turning cabin).

2 c) The Main Processing Computer. Figure 1 shows the architecture of the driving simulator system. Each component listed above is described below in detail. Figure 1. Architecture of the turning cabin driving simulator 2.1 The Image Display System The image display system displays the simulation image to drivers and thus plays an importantt role in enhancing the realism of the driving simulator. The goal of building the image display system is to create a compelling visual display which increases the feelings of presence for drivers. Large FOV displays are now part of by most mid-level driving simulators. Some simulators use a set of separated flat screens, typically a left screen, a front screen and a right screen, to provide drivers a virtual environment with a large FOV. A significant disadvantage of using multiple flat screens is the discontinuouss projection of the image on screens. The discontinuity of the image is noticeable at the part where the screens interconnect. A cylindricall screen was used in our driving simulator to solve this problem. Threee LCD projectors are tiled together to produce a 180-degree FOV display. Figure 2 displays a 3D view of the multi-projector display system. Figure 2. A 3D view of the multi-projector display system To produce a usable seamless display with multi-projectors, a variety of image corrections are required. First, geometric distortions resulting from projecting onto the non-planar (cylindrical) surface must be corrected Geometric Distortion Correction

3 A number of techniques have been developed to correct geometric distortion [4][5]. An approach proved flexible and effective is named as 2-pass rendering algorithm [7]. To render correct perspective imagery on irregular surfaces, this approach computes and stores the desired image as a texture map in the first pass. In the second pass, the approach projects the texture from the user s viewpoint onto the polygonal model of the display surface. The desired image texture is then mapped onto the display surface model and rendered from the projector s viewpoint. The result of this approach is a pre-distorted image which appears undistorted from the user s viewpoint. In our case, we developedd a simplified algorithm thatt calculates only the cylindrical correction and allows manually adjustments. Although our algorithm lacks flexibility and some accuracy, the distortion correction has proved acceptable with no assistances from external devices. The fact that the driver sits at the center of the cylindrical screen and only rotates along the yaw axis guarantees a static user s viewpoint. First, by mounting each projector onto a static base and properly placing and rotating the projectors, distortions due to misplacement can be partially eliminated. Instead of calculating calibration information from pictures taken by cameras, we compute the calibration information by introducing some variables. By changing values of these variables, one can adjust the pre-distorted image till it appears un-distortedd on the cylindrical inner surface. Factors that would affect the geometric distortions are: 1) the perpendicular distance of the projector to the screen surface, 2) the height of the projector above the ground, 3) the tilt angle of the projector and 4) the diameter of the cylindrical screen. By looking into the projector specifications, the projection size is easily calculated. Figure 2 illustrates a single projector projecting onto the inner surface of the cylindrical screen. Figure 3. A 3D view and a side view of the image display system As shown in Figure 3, instead of the virtual planar surface L1-R1-H1-R2-H2-L2, the image is projecting onto a curved surface with the left boundary L1-L2, the right boundary R1-R2, the top curved boundary L1- H1 ' -R1, and the bottom curved boundary L2- H2 ' -R2. The pixel at point P is projected at point P onto the cylindrical screen surface. To correct the distortion, the projected position of each pixel on the cylindrical screen should be computed. Figure 3 shows the projection of a pixel (m,n) ontoo the cylindrical screen. As shown in figure 3, the perpendicular distance of the projector to the screen is dp, the height of the projector is hp, the tilt angle of the projector is and the radius of the screen is r. The pixel (m,n) hits the virtual planar screen at point P and hits the cylindrical inner surface at point P. The resolution of the image is w h. The ratio of y coordinates of P and P is given as below: (1) where: a=(r/2-mr/w). Therefore, to correct the image distortion in the vertical direction, a pre-distortion has to be applied. Define the pre- distorted y coordinate of pixel (m,n) as h2, and we have:

4 hence h2 h1 (2) Multiplying y coordinates of each pixel by h2 h1 h1 (3) results in the pre-distorted y coordinates of all pixels. The image is not only distorted along the vertical direction, but also the horizontal direction. When projecting onto a planar screen, we have the horizontal distances between adjacent pixels as: (, ) (, ) (, ) (, ). However, when projecting onto the cylindrical screen, the horizontal distances between adjacent pixels are not equal across the screen as shown in figure 4. To correct the distortion, we have to equalize the horizontal distance between adjacent pixels Cylindrical screen inner surface planar surface distance between two adjacent pixels pixel Figure 4. Distances between adjacent pixels on the cylindrical screen compared to that on the planar screen. Similar to the computation of y coordinates of P and P : (4) Divide the cylindrical into (w-1) equal sections along the curved surface. The pixel (m,n) should be projected onto the screen so that the length of the arc S P π ( ). Therefore, ( ) w1 should be equal to r π( ). To correct the distortion, a ( ) pre-distortion is applied. Define the pre-distorted x coordinate of pixel (m,n) as w2, and we have: w2 rπ( ) (5) ( ) hence w2 r π( ) ( ) rπ( ) ( ) (6) A pre-distortion mesh is generated according to the calibration information computed above. Mapping the image to the pre-distortion mesh and projecting the result onto the cylindrical screen produces an undistorted image from the driver s viewpoint. The diagram of applying the geometric distortion correction is shown in Figure 5.

5 Figure 5. Pre-distortion applied to produce an undistorted image on the cylindrical screen Edge Blending Another image correction needed in a multi-projector display system is photometric correction. Photometric correction deals with two kinds of circumstances: 1) non-uniformity among projectors and 2) non-uniformity within each projector [4]. This study focuses on the former issue. Multiple projectors are combined together to display a single image by each projecting a part of the image source. It is considered a low-cost and effective approach to produce a bright and high-resolution display. However, it s found almost impossible to manually align multiple projectors in a pixel perfect way so that there is no gap between the images or a bright seam. The problem of misalignment can be a significant problem for driving simulators because objects often move acrosss channels. A method thatt is widely adopted to solve the problem is to overlap adjacent images. A consequence of overlapping images is the brighter overlap regions, which is quite noticeable to users. To make the overlap region appear seamless, we use edge blending techniques. Below are the steps to implement the edgee blending technique in this study: 1) Overlap adjacent projected images by a proper pixel amount. The overlapped regions are over illuminated as expected. 2) Apply the blending by fading out the image from one projector in the overlap region, while fading up the image from the adjacent projector in the overlap region. This is done by multiplying the pixel of the right image in the overlap region by the blend function f(x) and the pixel of the left image in the overlap region by 1-f(x). The sum of the values assigned to a pixel in one image and its corresponding pixel in the adjacent image must be 1. Below is the blending function f(x) developed by [10]: (7) Figure 6. Blend function of two adjacent images and the mask created for the center image

6 where P and a are constants. The overlap region is normalized on the interval [0,1], while 0 refers to the left most pixel, and 1 refers to the right most pixel. A graphical look of the blending function and created mask is shown in figure 6 3) Blend the mask with the image source before projecting onto the screen. Figure 7 shows the image projected on the cylindrical screen before and after the implementation of edge blending. Figure 7. Comparison between the images projected on the screen before and after applying edge blending 2.2 The 1 DOF Driving Simulator Platform (Turning Cabin) The platform is composed of two components linked mechanically: 1) the motion control system and 2) the cabin. The motion control system is used to control yaw motion of the platform. It contains a digital servo drive and an AC servo actuator with a reduction ratio of 160. The digital servo drive receives a command signal from the computer and transmits electric current to the AC servo actuator after amplifying the signal to produce motion proportional to the signal. The command signal sent from the computer represents the desired position, velocity or torque of the actuator. The digital servo drive monitors feedback signals from the actuator and continually adjusts for deviation from expected behavior. The communication between the computer and the digital servo drive is performed through the RS232 port in ASCII data format car heading (degree) time (s) Figure 8. Rotation of the actuator compared with the virtual vehicle heading angle The computer updates the states of the virtual vehicle with inputs from gas/brake pedals and the steering wheel. The heading angle of the cabin should be updated so that there is not a mismatch between the driver s heading and the

7 perceived heading from the visual display. As discussedd above, the rotation of the actuator controls the yaw movement of the cabin. The computer sends the heading angle of the virtual vehicle in ASCII format to the digital servo drive continuously. The AC servo actuator, driven by the digital drive, has a speed and acceleration rate of and respectively at the peak current. Figure 8 shows the actual rotation angle of the actuator and the virtual vehiclee heading angle sent from the main computer. The time lag between is about milliseconds. We are examining possible causes of this lag and hope to reduce it. The cabin is mounted on the AC servo actuator and rotates as the actuator moves. It is equipped with gas and brake pedals, and the force feedback steering wheel which allow acquisition of the driver s commands with sensors. The computer reads in these inputs and updates the states of the virtual vehicle based on the vehiclee dynamic model. The functional scheme of the motion platform is shown in figure 9. Figure 9. The functional scheme of the motion platform 3. EXPERIMENTS This section presents the results of experiments done with the driving simulator for the evaluation of optical flow patterns perceived and its relationship with the degree of simulator sickness reported. Twelve subjects, whose age ranges from 18 to 21, participated in the study. The experiment is divided into two runs: 1) using the turning cabin simulator and 2) using the fixed-base simulator. Each subject drove both runs. Half of the subjectss were given run 1 first, and the other half were given run 2 first. Upon arrival in our laboratory, participants filled out a simulator sickness questionnaire (SSQ). They were given the SSQ again after the completion of run 1 and run 2. There is a 5 minutes break for participants between two runs. Figure 10 presents an example of the optical flow field experienced by a subject when making a left turn in the simulator [10]. Each optical flow vector shows the magnitude and direction of optical flow. Figure 10. The optical flow field experienced by a subject in the simulator when making a left turn

8 Figure 11 shows the quantity of optical flow per second at a left and a right turn by drivers when using both simulators. For all drivers, the amount of optical flow is greater when making right turns then left turns. For both left and right turns, the amount of optical flow when using the turning cabin simulator is less than that when using the fixed base simulator. The magnitude of this difference is greater for right turns than for left turns. Figure 11. Quantity of optical flow experienced by 12 drivers per second when making left/right turns Figure 12 shows the means of the amount of optical flow per second for 12 participated drivers when making left/right turns in both the fixed base simulator and turning cabin simulator. It reveals that drivers experienced a higher amount of optical flow in the fixed base simulator. Also, the amount of optical at right turns is significantly higher than that at left turns, for both types of simulators. Amoutn of optical flow ( E05) fixed base simulator turning cabin simulator left turns right turns Figure 12. Means of the amount of optical flow per second when making left/right turns by type of simulator Figure 13 presents the average SSQ scores for the 12 subjects before run 1, after run 1 (using the turning cabin simulator), and after run 2 (using the fixed base simulator). Using the Wilcoxon Matched Pairs Test, we found the difference between these scores to be significant at the less than.1 level of confidence. A higher degree of simulator sickness was reported after Run 2 (using the fixed base simulator) Before Run 1 After Run 1 After Run 2 Figure 13. SSQ scores by run order

9 4. CONCLUSIONS We built a turning cabin driving simulator to decrease the amount of optic flow experienced by drivers as compared with other simulators. Multiple commodity projectors were used together with a 360-degree cylindrical screen to produce a high resolution and seamless display. Pre-distortion of the image was computed and applied before projecting onto the cylindrical screen. This corrected the geometric distortion. To create a smooth image across the display, edge blending was applied to eliminate the gaps between images and over-illuminated overlap regions. Both geometric distortion correction and edge blending were done via software. The 1 DOF platform made it possible to produce yaw motion for the driver s cabin. It helped to enhance the realism of the simulator by giving drivers a feeling more similar to a real world driving experience. More importantly, as revealed from experiments done with the turning cabin simulator, drivers reported a significant lower amount of simulator sickness when the simulator cabin turned along the yaw axis. Therefore, the result of this study suggests that adding the yaw motion component to a driving simulator is a potential way to compensate the undesired simulator sickness caused by a large FOV display. A possible reason why less simulator sickness was reported with the cabin turning, the amount of optical flow experienced by drivers in the simulator was analyzed. Results showed that a much higher amount of optical flow was experienced by all drivers when the cabin was static. REFERENCES [1] Arioui, H., Hima, S. and Nehaoua L., 2 DOF Low Cost Platform for Driving Simulator: Modeling and Control, Proc. IEEE/ASME International Conference on Advancd Intelligent Mechatronics, (2009). [2] Chiew, Y.S., Abdul Jalil, M.K., and Hussein, M., Motion Cues Visualization of a Motion Base for Driving Simulator, Proc. IEEE International Conference on Robotics and Biomimetics, (2008). [3] James Jeng-Weei Lin, Henry B.L. Duh, Donald E.Parker, Habib Abi-Rached, Thomas A. Furness, Effects of Field of View on Presence, Enjoyment, Memory, and Simulator Sickness in a Virtual Environment, Proc. IEEE Virtual Reality Conference, (2002). [4] Li, C., Lin, H., and Shi, J., A Survey of Multi-Projector Tiled Display Wall Construction, Proc. 3 rd International Conference on Image and Graphics, (2004). [5] Moriya, T. Beniyama, F. Utsugi, K. Minakawa, T. Takeda, H. Ando, K., Multi-camera and multiprojector based seamless live image display system, Proc. Multimedia Modelling Conference, (2004). [6] Murano, T., Yonekawa T., Aga, M. and Nagiri, S., Development of High-Performance Driving Simulator, SAE Papers, (2009). [7] Raskar, R., Brown, M., Yang, R., Chen, W., Welch, G., Towles, H., Seales, B., and Fuchs, H., Multi- Projector Displays Using Camera-Based Registration, Proc. IEEE Visualization Conference, (1999). [8] Seay, A.F., Krum, D.M., Hodges, L., Ribarsky, W., Simulator Sickness and Presence in a High FOV Virtual Environment, Proc. Virtual Reality, (2001). [9] Yoichi, A., Nobuyuki, U., Improvement of Driver s Feeling by Turning Cabin Driving Simulator, Proc. Driving Simulator Conference North America, Orlando, FL, (2005). [10] Yin,Z., and Mourant, R.R., The Perception of Optical Flow in Driving Simulators, Proc. The 5th International Driving Symposium on Human Factors in Driver Assessment, Training, and Vehicle Design, Big Sky, Montana, (2009). [11] local.wasp.uwa.edu.au/~pbourke/texture_colour/edgeblend/ [12]