Applications of Single Display Privacyware Phase 2, Project Design and Evaluation Specification

Transcription

1 Applications of Single Display Privacyware Phase 2, Project Design and Evaluation Specification Idin Karuei and Jeff Hendy 3 March 3008 EECE 518 Dr. Sidney Fels

2 Background Single Display Privacyware Single Display Groupware (SDG) describes the interaction technique in which multiple users interact with a single shared display. This is a good technique for collaboration, but it introduces several challenges. One important challenge is lack of screen space; no matter how large the display is, space will quickly run out when it is being used simultaneously by several users. Another concern is information overload to the users; when working collaboratively, there are certain things that should be seen by all users, but allowing every user to see everything will clutter the screen and make it difficult to focus on the task. Single Display Privacyware (SDP) [10] is a technique for interaction in which private information is displayed to individual users through a single public display. This technique has been explored by Shoemaker and Inkpen, and should help address many of the issues with SDG. Shoemaker and Inkpen implemented a demonstration of SDP with modified shutter glasses. Shutter glasses are usually used to present 3D stereo graphics to users. The glasses have LCD images, which can either be clear or opaque. When used with 3D stereo, the shutters alternate eyes at the same speed as the refresh rate of the display, allowing one eye to see only the even numbered frames and the other to see only the odd numbered frames. With proper graphics, this can create a very compelling 3D effect. Shoemaker and Inkpen used two pairs of glasses that were modified so that both eyes of one user see the even frames and both eyes of the second user see the odd frames. Using these glasses, Shoemaker and Inkpen created a software environment for building structures with virtual LEGO-like blocks. Both users could see the main work area, but each could only see her own cursor and context menus. Additionally, each user was able to look at a different set of on screen instructions for building specific figures. A user study showed that the SDP version of this program yielded strong improvement over an SDG version where both users saw the same image. Our Applications Obfuscation of Sensitive Information Although their interaction technique was called Single Display Privacyware, Shoemaker and Inkpen hid information from users not because it was truly private information, but because it wasn t needed by all users and would be distracting. We will now be

3 considering the case when a user needs to see information that he or she truly does not want to be visible to others because of privacy or security concerns. Berry, Bartram, and Booth [2] have demonstrated the need for this and proposed on solution. They illustrate the problem in the context of a presentation in which the presenter is showing a spreadsheet that contains private information. The presenter needs to view this information, but it cannot be shown to the audience. The proposed solution is to use two displays, one showing the full spreadsheet that only the presenter can see, and one in which the sensitive information has been masked for the audience. This works well in situations when two displays are readily available, but it is not always possible or desirable to have two separate displays. For example, the presenter might need to gesture to certain portions of the display, and this would be difficult if she had to continually return to her personal display to see all of the information. Some situations such as mobile devices or tabletop computers also do not lend themselves well to having a second display. Our approach to this problem will be to present the sensitive information in the even numbered frames while adding noise to the odd numbered frames. With a high enough refresh rate, the private information should be completely obscured and undecipherable to the naked eye. However, if the presenter is wearing shutter glass that filter out all of the noise in the odd frames, she should be able to clearly see the sensitive data. There are potential ways for malicious individuals to gain access to the hidden information. For example, an audience member could bring shutter glasses and synchronize them to the display or capture high speed video of the screen so that the information can be extracted from the appropriate frames later. Thus, this technique might not be appropriate for highly sensitive information in areas where untrustworthy people may be present, but it should suffice for typical applications such as in a classroom or in a meeting where all parties are trusted. 3D Interaction 3D displays can be useful in many situations, including design environments. 3D stereo with shutter glasses is one of the best ways to achieve 3D effects, but unfortunately, it is not possible with the modified glasses that are necessary for SDP. Another popular way to achieve 3D perception is with Fishtank VR, in which the position of the users head is tracked so that the perspective of the 3D graphics can be changed to follow the user around the display. Fishtank VR is generally limited to only one user because multiple users would require multiple viewpoints, and standard displays can only show one image at a time. We will attempt to combine Fishtank VR with SDP so that two users can simultaneously interact with a 3D environment using a single display. We will implement an example program in which two users must collaborate to drill holes through a 3D object while avoiding

4 obstacles that are present on each side of the object. Thus each user must be looking at one side of the 3D object while discussing what the other is seeing in order to find an appropriate location. Related Work Many related technologies have been introduced in the areas of personal displays and multiple views of 3D displays. Each has unique advantages and disadvantages, and we feel that our solution is optimal for the SDP task that we have described. Kitamura et al.[4] have created a system that allows multiple users to view different perspectives of a 3D stereo image from a single display. This is achieved by placing a mask above the screen with a circular cutout so that each user can only see a portion of the screen. Each user wears shutter glasses, and as long as the users are properly positioned so that their viewing areas don t overlap on the screen or go off the edge of the screen, they can see a full 3D stereo image. This approach has many limitations, including the fact that each user can only see a very small portion of the screen and much of the screen (the area between the users viewpoints) is completely wasted. Additionally, the movement of users is severely restricted because they must ensure that their viewing areas do not overlap. Our SDP/Fishtank hybrid will allow both users to make use of the full screen with the disadvantage of removing the 3D stereo capabilities and limiting use to two people. We feel that this is an appropriate tradeoff for the situation we are targeting. The Lumisight Table [7] is a tabletop display with a special layer on top called Lumisity Film that redirects light to different users. The display uses four separate projectors in conjunction with the film to present unique viewpoints to users at each side of the table. The disadvantages of this are that users cannot leave their side of the table and the need for four projectors drastically increases cost and complexity. Additionally, the Lumisight Table only provides 2D interaction. Agrawala et al. [1] have demonstrated the use of 3D stereo with different perspectives for two users. They modified shutter glasses so that each of the four total eyes would see only one of every four frames of the output. The 3D effect was successful, but due to limitations of current display technology, the slow shutter speed on the glasses produces a strong flicker effect that renders this concept useless for real applications.

5 Design of Obfuscation Software Goal We will attempt to create a system that can successfully hide information in such a way that it is visible in its entirety to somebody who is wearing properly synchronized shutter glass, but is completely hidden to the naked eye. Technology For the image obfuscation portion of this project, we will be using 3D shutter glasses with a stereo JPG viewer to display the obfuscated images on a standard CRT monitor. A stereo JPG file is simply a JPG image divided into two parts, with the left half of the image intended for the right eye, and the right half of the image intended for the left eye. When opened in a stereo JPG viewing program, the halves of the image are split and displayed on alternating frames of the video output. When viewed with properly synchronized shutter glasses, a well designed stereo JPG file will appear to be a 3D still image. Rather than creating 3D images, we will create JPG files in which one frame contains the private information while the other contains noise inducing information that will make the image undecipherable to the naked eye. If the private information is intended for the right eye, the left eye of the shutter glasses must be covered up so that the noise inducing information is not visible to the user. Images We will create a wide variety of image types for evaluation. The most simple is a seven segment number, as shown below.

6 Figure 1 - obfuscation example for 7 segment numbers In this example, the even numbered frames will contain the number 5, while the odd numbered frames will contain the unused segments from the number 5. With these two images alternating rapidly on the screen, the naked eye should see them combined into the number 8, which somebody wearing shutter glasses with the left eye covered will see the true data. After completing examples with seven segment numbers, we will work on ways to obfuscate more complicated images including text, basic shapes (square, triangle, etc), and detailed images in both black and white and colour. Design of 3D Drilling Simulation The Environment - Graphics Everything happens in a 3D virtual environment that is built by OpenGL in C++. The environment is a rectangular tunnel that starts in front of the display (between the display and the users) and goes far behind the display as pictured below.

7 Figure 2 The display is like a window to this virtual world. Any object that is located inside the frame from the view point, whether it is behind or in front of the display, will be seen and objects that are not inside the frame will not be seen; this describes the field of view. There is no need to put more effort into hiding the stuff that are not in the field of view because they will automatically be taken care of by being out of the screen. A wall divides the whole tunnel into two regions on the left and the right, i.e. it starts in front of the display and cuts the display and goes to the back of the display. The wall will block the light too (unless using an opaque wall is desired for different applications), therefore it is logical to have two sources of light, one on the left and one on the right side of the wall. Perspective Fishtank VR As a user moves around, the 2D screen adapts to his/her position by displaying the 3D environment as it should be seen from that point of view. A head tracker (positioned on the head of the user) is used to input the coordinates of the view point to the software. So far, everything has just been the same as fishtank VR. If someone else looks at the display, everything would look very weird. This new person will see the environment from the perspective of the first user. This is one weakness of the regular fishtank VR which will be improved in our design.

8 Sharing the Screen Shutter Glasses In our design the two users see the environment each from his/her own perspective. This can be achieved by sharing the display in time domain. Any display shows stationary or moving pictures by continually refreshing the display. A number of pictures (frame rate) are shown at each second. We divide them into two sets: even frames and odd frames. The environment from the perspective of user 1 is shown on the odd frames and the environment from the perspective of user 2 is shown one the even frames. The two users will then see both perspective views overlapped on each other with 50% transparency which is not desired. We should now block user 1 from seeing perspective 2 and vice versa. The shutter glasses are used for this reason. User 1 wears a pair of shutter glasses that block the even frames and user 2 s shutter glasses block the odd frames. Since the shutter glasses available in the market are made for stereo viewing, the left and the right eye filters flicker at the same frequency with a 180 degree difference in phase which means at any time only one of them is blocking the light. If we use those glasses with no modifications each user will see the environment from his perspective on one of his/her eyes and what the other user should see on the other eye. In order to solve this problem one may change the wiring of the shutter glasses. Instead of modifying the glasses we will just blind fold one eye of each pair of glasses. Figure 3

9 Definitions For simplicity we will define important terms that will be used frequently in the rest of this document before going any further. Driller: the one who has the drill and should make a hole in the wall by clicking on his/her side of the wall. Helper: the one who is on the other side of the wall and helps the driller complete the task just by giving directions. Front Side: the side of the wall visible to the driller. Back Side: the side of the wall visible to the helper. Front Region: a region on the front side where it is safe for drilling. (The red rectangle in Figure 4-b) Back Region: a region on the back side where it is safe for drilling. (The blue rectangle in Figure 4-a) Task: a one time drilling job. Only one of the users has a drill, and should make a hole in the wall and vertical to the wall. The drill should enter the wall inside the front region and exit the wall from the back region. Therefore the whole should be in the intersection both regions. (The purple rectangle in Figure 4-c) Trial: a set of n tasks in one condition (2D or 3D). Series: a set of 4k trials in a specific order. Error: if the hole is not in both the front and back region an error has occurred. Drilling Input The task as described above, is to drill a hole in the wall. The driller has the complete control over the job by using a mouse. The cursor is always on the front side of the wall, visible to the driller (if positioned correctly). By moving the mouse to any direction, the cursor moves on the wall to the same direction from the perspective of the driller. By hitting the left click of the mouse the driller chooses the location and drills into the wall. The delay time relates to the time spent to drill completely. The driller may not know if he drilled right or wrong until the drilling is completed. If the task is successfully done another task will be given to the users but if the task is done with error they should redo it.

10 Figure 4: a) front region, b) back region, c) target Comparison 2D Interaction For evaluation purposes, we will be creating a similar design tool in 2D. The goals, rules, and terminology will be the same, but instead of using Fishtank VR and shutter glasses, the two users will be looking at the same 2D screen. The front and back side will be show next to each other.

11 Figure 5 - Screen Mockup of 2D Design Task Evaluation We will conduct evaluations for both the obfuscation and 3D design portions of the project as well as the overall experience of using shutter glasses for work other than stereo 3D. Participants We will attempt to recruit participants, yielding 5-7 pairs for the collaborative design task, and individuals for the obfuscation task. For each task, we will be collecting quantitative data for statistical analysis as well as qualitative data to get a sense of the participants feelings about the system. Since both parts of the project are highly visual, we will attempt to recruit participants with normal vision or fully corrected vision. Other demographic characteristics are not relevant to our study, though we will attempt to recruit a wide variety of participants in case any unexpected trends emerge. Obfuscation The obfuscation task will be evaluated with each participant individually. We will first explain the concept to the participant and then show an example screen containing

12 various obfuscated images, allowing the participant get comfortable viewing them with the shutter glasses and without. After this introduction, the participant will be shown several screens containing obfuscated images with different qualities. As discussed in the design portion, there will be four different types of images (7 segment numbers, text, shapes, and detailed images). For each type of image, one screen will be created with a grid of images. Each row of images will represent different colour complexities, moving from grayscale to solid primary colours to full colour images. Each column will represent a different contrast level moving from full contrast (ie, black on white) to low contrast (ie shades of grey with barely perceptible difference). For each screen, the participant will first be asked to identify what is shown in each individual image without the use of shutter glasses. We will record which images are able to be successfully identified. If any images are identified, we will ask the participant to rate how difficult it was to identify the image. Next, the participant will view the screen with the shutter glasses, and we will again record which images are able to be successfully identified and whether any difficulty was experienced identifying the image. Using this data, we will be able to determine which complexities of images are able to be successfully obfuscated when not wearing the glasses and successfully identified while wearing the glasses. We will additionally be able to determine what effect the use of different colours and contrast ranges have on the obfuscability of images. After completing these tasks, the participants will be interview about their experience. Specific questions include:! Did you experience any headaches or other discomforts using the shutter glasses?! If you were using this system, would you feel confident that your personal data is safe?! Would you be comfortable with wearing shutter glasses for an extended period of time in order to protect your private data? We will then engage in open ended conversation with the participants to discover other opinions regarding benefits and shortcomings of this system. 3D Drilling Simulation Evaluation of the 3D design consists of two parts: quantitative measurements comparing the 3D interface to the 2D interface, and a survey following that. The measurements should give us a solid scientific basis for any improvements that we might find, but there are a number of issues to explore that cannot be measured so easily. One issue is that when pointing to the object in 3D space, one user s finger could completely obscure the view of the other so that the 3D effect is lost. We are additionally interested in knowing

13 whether the shutter glasses will cause any discomfort or queasiness during the experience. Each group of two people will be given a series of trials. The series contain k trials in 2D condition and k trials in 3D condition. For each trial T2 i that is done in 2D environment there is a trial T3 i in 3D with exactly the same tasks and in the same ordering. This is because we want to make sure to compare exactly the same tasks. In order to eliminate the effect of learning, we have to keep a balance between the 2D trials that are done before their 3D counterparts and the 2D trials that are done after their 3D counterparts. For example: the series S 1 ={T2 1,T3 2,T3 1,T2 2 } is desired because one trial is done in 2D before 3D and another is done in 3D before 2D but S 2 ={T2 1,T2 2,T3 1,T3 2 } and S 3 ={T2 1,T3 1,T2 2,T3 2 } are not desired because in both of them the 2D trials are done before the 3D trials which might make the 3D trials easier because the users may get used to them during the 2D trial. This means that we have to test an even number of 2D trials and the same number of 3D trials, half of the 2D trials before their 3D counterpart and half of them after their 3D counterpart. This adds to a number divisible by 4 (4, 8, 12, trials). The time and the number of errors will be measured for each trial. For simplicity and accuracy of the measurement the time will be measured from the end of the first task to the end of the last task. This first task of each trial will not have any effect on the measurements. In case of the occurrence of an error, the group will have to repeat the task, but in order to avoid multiple clicks the next task (which might be the same task if an error occurs) appears after a delay time. During the delay time the current completed (successfully or with error) task remains on the screen. If an error occurs, the color of the region changes (from blue to red or red to blue) as an indicator. If the user attempts to redo the task during the delay, the system gets the input and keeps it in a buffer until the end of the delay time just as if the task is done at the end of the delay time. In Figure 6 you can see the flowchart of this process. During the 3D trials, the users will wear the glasses and sit beside each other looking at the display with a 45 degree angle; they will see the 3D image of the wall and the regions and the driller can move the mouse cursor on the wall to choose the drilling point. During the 2D trials, the users will sit beside each other looking straight at the display. The two sides of the wall will be shown in 2D beside each other on the left and right of the screen and the driller should choose the drilling point on the front side. We will conduct statistical analysis on the time measurement to expose any difference between the two interfaces. Before rushing into the actual study, a pilot test should be done to tune the number of tasks in each trial, the number of trials in each series, and other factors to make use of the time in the most efficient way during the tests.

14 Figure 6 - The Flow of Tasks in Quantitative Design Evaluation Survey Following the series of tasks, a questionnaire will be given to both users in each team to gather more information about their ideas and their experience with the environment. The questions will include:! Does the finger get in way? (If answered no, go to question 3)! Can you work around it?! Do you like interacting with the environment?! Which one do you prefer, working in the 3D environment or the 2D?! Do you feel any headaches or other side effects?! Do you think you may get sick if you use the 3D environment for long?! Does the 3D environment help you cooperate? We will then engage in open ended conversation with the participants to discover other opinions regarding benefits and shortcomings of this system.

15 References [1] Agrawala, M., Beers, A. C., McDowall, I., Fröhlich, B., Bolas, M., and Hanrahan, P. The two-user Responsive Workbench: support for collaboration through individual views of a shared space. In Proceedings of the 24th Annual Conference on Computer Graphics and interactive Techniques International Conference on Computer Graphics and Interactive Techniques [2] Berry, L., Bartram, L., and Booth, K. Role-Based Control of Shared Application Views. UIST [3] Fitzmaurice, G. W. Situated information spaces and spatially aware palmtop computers. Commun. ACM 36, [4] Kitamura, Y., Konishi, T., Yamamoto, S., and Kishino, F. Interactive stereoscopic display for three or more users. In Proceedings of the 28th Annual Conference on Computer Graphics and interactive Techniques SIGGRAPH ' [5] Leung, Y. K. and Apperley, M. D. A review and taxonomy of distortion-oriented presentation techniques. ACM Trans. Comput.-Hum. Interact [6] Mackinlay, J. D., Robertson, G. G., and Card, S. K. The perspective wall: detail and context smoothly integrated. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems: Reaching Through Technology [7] Matsushita, M., Iida, M., Ohguro, T., Shirai, Y., Kakehi, Y., and Naemura, T. Lumisight table: a face-to-face collaboration support system that optimizes direction of projected information to each stakeholder. In Proceedings of the 2004 ACM Conference on Computer Supported Cooperative Work [8] McDowall, I. E., Bolas, M. T., Hoberman, P., and Fisher, S. S. Snared illumination. In ACM SIGGRAPH 2004 Emerging Technologies [9] Rao, R and Card, S. The Table Lens: Merging Graphical and Symbolic Representations in an Interactive Focus+Context Visualization for Tabular Information. Proceedings of the SIGCHI conference on Human factors in computing systems [10] Shoemaker, G. B. and Inkpen, K. M. Single display privacyware: augmenting public displays with private information. CHI ' [11] Szalavári, Z., Eckstein, E., and Gervautz, M. Collaborative gaming in augmented reality. In Proceedings of the ACM Symposium on Virtual Reality Software and Technology