CORE OPERATING SYSTEM ELEMENTS FOR EMBEDDED INTERFACES

Transcription

1 SCHOOL OF INFORMATION TECHNOLOGIES CORE OPERATING SYSTEM ELEMENTS FOR EMBEDDED INTERFACES TECHNICAL REPORT 664 ANDREW JOHN CLAYPHAN AND JUDY KAY DECEMBER, 2010

2 Core Operating System Elements for Embedded Interfaces Andrew John Clayphan and Judy Kay School of Information Technologies University of Sydney Sydney, Australia, 2006 Abstract. The growing availability of relatively low cost hardware for Single Display Groupware (SDG), in the form of tabletops and large wall displays, is one of many emerging elements in a person s digital infrastructure. This creates many opportunities and creates new user interface challenges. An important one of these, that has had little attention, is the nature of the user interface for core operating system functions. This should support user tasks such as authentication, finding, and organizing files, coordinating the tabletop and the rest of each individual s digital ecosystem, supporting collaboration, starting applications, and customizing the appearance of the tabletop. This paper presents an analysis of the nature of tabletop interaction, identifying its distinctive characteristics and uses. Shown is how this affects the core operating system functions needed by tabletop users, both in terms of the ways this differs from desktops and in terms of how this defines the character of the interface to the functions. The key contribution is to provide a foundation for the principled design of user interfaces to support core operating system functions at tabletops. 1 Introduction We are seeing the emergence of a range of new interaction devices that offer the promise of a rich digital ecology of devices for user interaction. Among classes of widely available devices are the smartphone, a personal but powerful device, the laptop and desktop computers, also increasingly powerful personal devices. We are just beginning to see a new class of device added to a person s digital ecosystem. These are the tabletops, also called single display groupware devices. These create new possibilities for people to work together at a single device [62]. This makes them fundamentally different from devices such as a desktop. Reflecting the distinctive nature of tabletops, we have already seen considerable research on many aspects for supporting tabletop interaction, including, for example, exploration of natural gestures for touch interaction [17,69,71], territoriality that affects management of the space on the table [45,58,65], interaction in

3 the space above the table [23,32], new approaches to creating menus [29,30] and ways to integrate use of physical objects [11,12,21]. However, there has been very little work to explore the nature of the fundamental facilities, the core concepts underlying the interface to the operating system that need to be available to support the effective use of tabletops. With the fast emergence of new tabletop hardware, supporting natural interaction, particularly touch-based interaction, it is timely to explore this issue. This paper tackles the task by a careful analysis of the nature of tabletop interaction, the aspects that distinguish it from interaction with other devices such as desktops, and then reasoning about the core concepts that should be supported within the interface to the operating system when using tabletops. Fig. 1. The role of core concepts for the user interface to the operating system Figure 1 illustrates the notion of the core user interface to an operating system. The figure shows the actual operating system at the bottom. (Of course, below this is the actual hardware). At the top is the layer that the user actually sees; in the case of desktops, this is generally the interface provided by the windowing system. The middle layer, the core concern of this paper, is the conceptual elements of this user interface which represent applicable things and actions, which we call Thacts. So, for example, a user coming to any new desktop system would expect a file system which makes it possible to work with files, where these might be images, documents, spreadsheets, web pages and many others. The user would also expect to be able to do tasks, such as starting an application, copying, renaming and deleting files. Current windowing systems are very alike in terms of both the upper two layers of the figure on all widely used desktops [40]. While this means that users of tabletops are likely to be familiar with a desktop user interface paradigm, the significant differences between desktops and tabletops mean that we cannot assume that this will work well for tabletops. Notably, we cannot expect the Thacts of the desktop user interface to support the special and different aspects of the tabletop. 2

4 Table 1 identifies three key classes of use of tabletops. The first row represents the use that has dominated research in tabletops, where the tabletop is assumed to be available for any user. The final row of Table 1 indicates the opposite extreme where the tabletop is for personal use in a restricted space. This has had little attention in research, although there has been one reported example of long term personal use [67] based on the Windows desktop interface and this pointed to major limitations of that way to use a tabletop. Class Public Space Restricted Group Use Restricted Private Space Example Kiosk Group planning, Classroom, Board Room, Home As a Desktop Table 1. Distinctive classes of use of tabletops The middle row describes an important emerging role for a tabletop, where it is embedded within a restricted space, for use by the people in that area. So, for example, an important role of the tabletop in a home would be to support the needs of the people who live there. We are already seeing many schools with interactive whiteboards that could be easily used as a tabletop. In this case, if the tabletop is in a room that is primarily used by a single class of students, it should be able to support collaborative tasks by small groups within that class. This class of use for tabletops is likely to be particularly important as it exploits the particular affordances of tabletops to support small group work, at an interface that is conveniently located in the environment used by the group. To date, reflecting the rather new nature of tabletops and the many associated interface challenges, there has been relatively little work on this role and even less exploration of the core interface elements of the operating system interface they should offer, with some notable exceptions [18,33,46,47]. This paper aims to create a foundation for the design of effective interfaces for tabletop interaction by identifying the core operating system elements that are needed for the range of roles for which tabletops offer most promise, notably the restricted group use in the middle row of Table 1. We drive our work with an analysis of; the nature of tabletops and their place in the user s digital device ecosystem; the distinctive classes of use and Thacts. This is followed by discussions and a conclusion regarding the elements we identify as important. 3

5 2 Analysis of the Nature of Tabletops and their Place in the User s Digital Device Ecosystem Our goal is to identify the Thacts for tabletops. We start by analyzing the particular characteristics of tabletops, their relationship to other devices that people use, their important similarities to these as well as their differences. In particular, we clarify key differences between tabletops and the extremely well established and widely used desktops. This is important for informing analysis of the ways that the Thacts for tabletops might maintain consistency with the familiar desktop and where this would jeopardize the effectiveness of the tabletop, by denying the full potential of its particular affordances. In Table 2, the columns present a set of key classes of devices in what we call the user s digital device ecosystem, which is the set of digital devices a person may use. The rows describe the dimensions we use to capture the different nature of these devices. Dimension Mobile Tabletop Wall Desktop Laptop Tablet of nature Phone Group Use Private Ownership Mobility Keyboard/ Mouse Other, Physical Artefacts Large Screen Size Reach Issues Legend feature is present, more ticks refers to level of emphasis feature is missing or minimal in nature Table 2. Comparison of key characteristics of devices in a person s digital ecosystem The first column shows tabletops, our focus. Next we have large wall displays because they share some of the characteristics of tabletops. The remaining columns (desktops, laptops, tablets and mobile phones) represent classes of devices that are already in wide use. These are important for several reasons. First, many people are likely to be familiar with some form of these. So, the design 4

6 of tabletop interaction should take account of the user s existing knowledge and expectations, based on these devices, especially when the tabletop is to be used in conjunction with them. We now discuss each row, analyzing the ways that it helps to define the distinctive nature of tabletop interaction. The first row, group use, is a fundamental dimension, characterizing tabletops. While there has been some study of a single user with a tabletop (for example, [67]), the potential to support small-group collaboration is one of its defining features. Indeed, the alternate name, single-display groupware, reflects this characteristic. As the table shows, it shares this characteristic with large wall displays and all the other classes of devices differ on this aspect, since all are primarily for use by a single user at a time. The second row, private ownership, is closely related to the first but takes the notion of private use further, to private ownership. The table shows the mobile phone is a particularly personal, privately owned device. The desktop, laptop and tablet are also shown as privately owned, but we acknowledge that this is at a weaker level. Certainly, desktops and laptops typically support the notion of login, with multiple user accounts. In spite of this, many of these devices are almost exclusively used by a single person. The third row, mobility, distinguishes the tabletop as an embedded device that is fixed in one location. This follows from its size. If the tabletop is to be large enough to be used collaboratively, it needs to be large enough for several people to sit comfortably around the screen. This makes it inherently less portable than any of the personal devices, particularly the phone. The laptop may be better seen as portable rather than mobile [70]. Importantly, the mobility of devices like laptops, tabletops and mobile phones suggests that the user may want to be able to use them in conjunction with a tabletop. The fourth row, keyboard/mouse reflects the different nature of interaction, with touch being part of the vision for tabletop interaction. While many tabletop systems have keyboards, both on the table interface (for example, [34,35]) or as physical keyboards (for example, [20,24]), tabletops generally favor direct interaction with the tabletop surface. This difference has been the focus of much research in exploring how to effectively manage it. The fifth row indicates the role of other, physical artefacts, such as paper documents, objects such as cups or jars, passports and mobile phones. This is a somewhat subtle aspect; it is not entirely obvious that the tabletop should operate in conjunction with such artefacts when the other devices do not. But the literature certainly includes considerable work exploring this dimension (for example, [15,22,26,27]). Part of the reason follows from the ways that people currently use ordinary tables, putting things on them. Another part is due to the ease with which vision-based tabletop technology can readily recognize 5

7 some objects. But broadly, it is strongly influenced by the other dimensions of the nature of tabletops, especially that they are fixed, rather than mobile. This means that it is practical to add hardware and software to support recognition of objects, capture the content of paper documents and integration of other physical devices such as knobs [12,22,43]. Yet another reason for integrating other physical artefacts follows from the challenge of designing effective interaction support within the limitations imposed by the lack of a mouse and keyboard. And, another reason for this follows from their nature as a device fixed in one place; people may bring devices such as mobile phones and other materials for their collaboration to that place. The sixth row, screen size, reflects working area. Tabletops have large screens which makes it feasible for multiple users to work at device at the same time. Importantly, with existing hardware, the large screen is not necessarily matched with the capacity to achieve high resolution as well [51]. One important approach to this issue is to create very high resolution on large tables [8,64]. For most hardware, size and resolution are competing goals, with the former often winning the trade-off. The last row, reach issues, are formed from considering how a person interacts with a device. The large size of tabletops, combined with the goal of direct interaction, without a mouse, creates the need to consider a person s reach. In this respect, the tabletop and large wall display have to deal with the fact that the user may not be able to reach parts of the display area that hold artefacts that they wish to use. The desktop and laptop avoid this because they make use of a mouse (and if the user did need to touch the screen, it is within reach). The tablet and mobile phone avoid this problem because the screen is small, reflecting their mobile nature. 3 Analysis of Distinctive Classes of Use There are useful analyzes that combine those in Tables 1 and 2. Figure 2 taken from Ackad [1] summarizes the relationship between embeddedness in public versus restricted spaces, against the degree of specialization versus general purpose use. Revisiting Table 1, a tabletop has a number of roles to offer, and the one that holds most promise is restricted group use. This points to the need to explore the effect of the distinctive aspects of tabletops for this use, as identified in Table 2: collaborative use by a group of people, creating the need to consider different orientations and problems with clutter; the context of the fixed location; direct input without a keyboard or mouse and problems associated with reach. 6

8 Fig. 2. A map of the specialization and user space of everyday technology Collaborative Interaction A key goal of tabletops is to facilitate and support collocated collaborative interaction. This requires the support of multiple parties either working together or separately at an interface. Given current hardware limitations the design of tabletop software should aim to support small groups and exploit current as well as any likely hardware possibilities. Context of Use Restricted group use needs to be designed such that there is no overarching ownership on the table. This is to facilitate group use rather than private ownership. Input Interaction The tabletop must accommodate natural interaction for it to be used. As a keyboard and mouse may not exist, elements designed need to be receptive to the input methods available. In this case, this is touch interaction, and thus appropriately sized target zones should be set up to facilitate input. Reach As the table gets larger it is necessary to understand that not all areas of the table may directly be accessible by a user. In this case, considerations need to be made in the design of interfaces to counter this issue. Orientation As the point of restricted group use to is encourage and facilitate interaction, artefacts should have the ability to be situated and oriented appropriately, such that all users can take in information being displayed. Clutter Clutter arises when several elements cover the tabletop making it hard to complete tasks. This issue may arise from the need to view a lot of information and when there are multiple users are working at a tabletop. Mechanisms are required to reduce or remove this issue. 7

9 4 Embedded Devices Embedded Devices are elements that are fixed (or have become fixed) in an environment. An environment defines the access for individuals and the tools present. An embedded device can be optimized to support functions required of an environment. From Table 1, the vision for tabletops is focused on restricted group use. The tabletop provides an interface that acts as the link allowing individuals and groups to carry out tasks. With restricted groups, assumptions are made to the types of people that will use the tabletop. These assumptions can then be factored into the design of the interface in addition to assumptions about the environment to support the particular user interests. This is illustrated with a few examples: Research Lab: facilities are required to support meetings. School: elements to encourage and enhance activity between students are required for teaching lessons. Corporate Showroom: functions to display product information and the ability to capture orders are required. The tabletop is part of a user s digital ecology. This is one part of a series of devices and technologies of a pervasive computing environment. It follows from the nature of embedded, that particular actions can be realized through the design of the environment around the tabletop. As a tabletop will not move, it makes sense to allow for specialized hardware such as cameras to be set up to augment the pre-existing tabletop interface. 5 Analysis of Thacts for Embedded Tabletops The focus of the tabletop is of the role of restricted group use which is the middle row of Table 1. The characteristics of tabletops, as captured in Table 2 are considered and then the Thacts that should be supported in the default environment of tabletops are identified. Things are identified first and then the Actions on them. 5.1 Things and their organization Table 3 summarizes both the basic Things and the structuring mechanisms for collections of them. It begins with users; this follows from the collaborative nature of tabletops. They need to support multiple users. As we focus on embedded tabletop, there should be some people who are regular users, who are known to the tabletop, because they normally occupy the place the tabletop is located, where this may be a workplace area, a classroom or a home. 8

10 Things Brief Description Literature Users People within the space the table resides in. [24] Digital Artefact (DA) DA Representative Digital information element, e.g. item, [5,21] image, video. Representation of a DA, e.g. icon. Shared Space Places where Thacts are available to other users. [59] Private Space Places where they are not available to other [59] users. Digital Ecosystem Arbitrary other devices, DA stores people use, e.g. phone, cloud and web. [15,31,55] Physical Physical objects for use with the tabletop. [12,26,27] Artefacts (PA) Controller Interface element for performing actions. [12,53,69] Long term store of DAs DA persistance between sessions of use. [24] Hierarchical Abstract organization of DAs. [13] Structure on DAs Tagged Structure on DAs Abstract organization of DAs. [21] Context Collections of DAs and Controllers in a state, at a time. [24] Settings Configurable aspects of the above elements. [1] Table 3. Summary of Things and their organization. Bold indicates elements which have important differences from conventional desktops. The final column gives examples of work concerning this aspect. Next in Table 3 is the notion of a digital artefact and its representative on the tabletop. This tightly coupled pair of concepts is not different from other devices, such as desktops. However, it combines the generalized notion of a file as well as any interactive element, such as an application. It is noted that a file is really concerned with low level implementation rather than a concept in a mental model [41,25]. A mental model is an explanation of someone s thought process about how something works in the real world. 1. So taking as an example, in some systems each mail item is kept in a separate file while other systems use one file to hold all the mail items in a folder; from the user s point of view, this 1 model 9

11 is not important and is not normally visible. In both cases, the user is supported in thinking about each mail item as a separate digital artefact. As indicated in the table, this may be any digital information element that makes sense to the user and it may include active artefacts such as movies. A file system is one core constituent of an operating system [50] and underlying file systems impose limits on the nature of digital artefacts. The notion of DA representative describes what the user can see at the interface; in terms of the user s mental model, this is a representative of the more abstract digital artefact. So, for example, in the case of a movie that is not playing, the DA representative may be a simple static icon. The next pair of rows in the table relate to shared and private spaces. These notions are particular to tabletops and follow from their use for collaboration. Users will, at least at some time, have the need for private spaces (such as manipulating their own content) and that should be supported [59]. The digital ecosystem for embedded devices is particularly important and represents a shift from desktops. Since people come to a tabletop to work together, that shared device will only be effective if people can, as needed draw upon their private information stores and ensure that the results of their work are available for private use later. There has already been some work exploring the use of carried mobile devices [15,55] for this role and for providing a controlling interface that protects private information [42] as well as the use of physical artefacts to link to cloud resources [31,34]. The role of physical artefacts is a novel aspect of tabletops. As already noted they can serve as controllers and as a connection to users digital ecosystems. Indeed, existing guidelines for tabletop design explicitly calls for integration of physical artefacts [57]. Notably, users may expect to be able to gracefully integrate physical documents with digital ones at the tabletop [21]. A controller provides an interface for digital artefacts to be acted upon reflecting user expectations. It can take several forms: 1. Gestures, have been a focus of research on tabletops (for example, [17,38,71]). Gestures are typically based on actions of one or more fingers, but can involve the hand [69], a stylus or any other physical artefact, such as a phone [53]. 2. Menus, are conceptually like those on the desktop windowing model, so make use of the same mental model [3,4]. 3. Special digital objects, which appear on the tabletop as an icon and this, too, has been used on desktops, for example where a printer icon is used for printing by dragging documents onto it. 4. Special physical objects, which are conceptually similar to their digital counterparts, but have a special role in tabletops, as reflected in the body of work that involves tabletops and tangibles [12,26,27]. 10

12 The next row in the table refers to the long term store of DAs. It follows that digital artefacts may persist after a session, having the ability to be used again at a later point in time. The next two rows look at the structure of digital artefacts at a tabletop [13,21,24]. All are common to desktops (although tagged approaches are less widespread) and so we conclude that a user will expect them to be available at a tabletop. The next row, context, is particular to tabletop interaction. It refers to the complete state of the environment, including all visible and stored digital artefacts at a point in time, where this includes their organization at that time. This seems particularly important for enabling a group of people to restart a collaborative task, with the table in the same state as when they last met to work on it [24]. It will also be important for reverting to an earlier state, for example when one of the people at the table performs an action that alters the state in ways that others at the table were still working on. The last row, settings defines any of the variable aspects of the above elements and is pivotal to application management [1]. Overall, this analysis points to several Things that are important for the effective use of embedded tabletop interfaces. Most of these have had little attention and certainly, the current state of research does not provide a coherent basis for creating software that will enable users to think and work in terms of these aspects. 5.2 Actions involving digital artefacts The analysis of the Things, focusing on the digital aspects identified in Table 3 is now refined. This section does not discuss controllers as they are the mechanism for achieving the Actions. While the analysis is limited to digital artefacts, it should be noted that physical artefacts are partly covered under the digital ecosystem. Actions associated with users Table 4 lists three actions to support multiple users at a tabletop. Ideally a tabletop will identify each person present as a basis for tailoring the experience, for example, ensuring the availability of the digital artefacts they use. However, common tabletop hardware [34,48,61] has poor support for automatically associating touch with a person. A notable exception is DiamondTouch [35]. A lesser, but useful approach is to at least provide software support for users to manually identify themselves, for example Hunter [24] provides a set of images representing known users. 11

13 Actions Brief Description Literature Identification Determining the people present and linking individuals [24,35] to actions at the interface. Identification is af- fected by hardware support. Authentication Verification of the identity of a person. [28,54,55] Group Coordination Support Support for controls such as sharing and ownership. [9,24] Table 4. Summary of Actions associated with users Identification Identification is the act of recognizing and naming an entity. Identification in this section refers to users rather than digital artefacts. Being able to identify users allows actions to be personalized. While most systems have failed in the hardware to support proper automatic identification, it is still an incredibly important aspect to tabletop research. Authentication Authentication is the verification of the identity of a person 2. Given the large screen of a tabletop, the physical security of the place can partly restrict who gains access to the tabletop. One has to assume that people within the space can make use of the tabletop. There are particular situations which demand authentication, for example connecting to a remote service. This poses a problem. It is not straightforward to enter a password at a tabletop as it is a public space. Some work to address this includes: Schoning s [54,55] use of biometrics, RFID tags and mobile phones and Kim s [28] exploration of inputting passwords on the tabletop itself, each with limitations in terms of time and effort. After examining the literature, mobile phones [53] have seen to be a viable option for entering sensitive information as the device is both private and personal to each user. Group Coordination Support There are a full range of activities that may help people collaborate more effectively at the tabletop [24]. In many cases, it is valuable to create mechanisms that support team work [52] such as visualizations of group activity, as in [9,66]. Another dimension of support for group work is making use of ideas from existing tools for collaboration, such as Trac 3, particularly its support for issue tracking and version control

14 Actions on digital artefacts, and groups of DAs, and their representatives A summary of actions on individual and multiple digital artefacts is presented in Table 5. The table is split into seven parts each grouping similar actions. Each part is separated by double horizonal lines. The first part is concerned with the manipulation of DAs through move, rotate and resize. The second part is concerned with the concept of copying on a tabletop. The third part is concerned with the removal of artefacts through hide and delete. These three parts collectively manipulate digital artefacts on the tabletop. These actions are present in many systems (such as [6,8,34]). The fourth part deals with the transformation of digital artefact representatives to digital artefacts themselves, such as enlarging an icon (the representative) to reveal the digital artefact. The fifth part deals with organizational structure. The sixth part deals with browsing artefacts and the last deals with the concept of searching on the tabletop. Manipulation of Digital Artefacts People create, use, access and modify digital artefacts on the tabletop. The familiarity of desktop window managers such as Windows, Linux and the Macintosh motivate expectations for these primitives on tabletop systems. A desktop window manager currently provides mechanisms to move, copy, hide (minimize) and delete files. This is indicated by the Builds on the Desktop Mental Model column in Table 5. The ticks indicate Desktop experience meaning they are likely to be familiar with these concepts. The crosses indicate actions specific to the Tabletop. Many of these additional primitives are due to the collaborative nature of tabletops, this is shown on the rightmost column of Table 5. An explanation of the manipulable actions: Move: to change location. Move is essential because people need access to artefacts independent of where they are relative to the tabletop. Even with move, people still may not be able to reach DAs at distant parts of the table. Rotate: to turn around a point. Rotate is important for tabletops because people may be situated differently a tabletop. This creates many cases where it is useful to re-orient a DA (or its representative). Re-orienting an artefact allows it to be viewable at the right orientation. A tabletop is unlike a desktop in this respect. Users of a tabletop may move around. Tabletop interfaces should support this. In this case, also, reach may be an issue. Resize: to change dimension. When a group of people at a tabletop focus on particular DAs, it may be helpful to make these quite large. Conversely it may also be useful to make DAs small to minimise clutter and shift focus to other DAs. 13

15 Actions Brief Description D.M.M. 4 Lit. 5 Collab. 6 Move DA Rotate DA Resize DA Copy DA Copy linked DA Hide DA Delete DA Transform DA representative to DA Alter the location of DA(s), or their representatives. Alter the orientation of DA(s), or their representatives. Alter size of DA(s), or their representatives. Create multiple copies of DA(s) for individual use. Create multiple copies of DA(s) for collaborative use. Hide DA(s), or their representatives. Remove DA(s), or their representatives. Convert a model into an artefact, such as starting an application. [19] [49] [7,19] [49] [7] [49] [7] [36] [6,7] [6] [7] [34] [36] [1] Group DAs Any form of grouping of DA [44,56] representatives. Alter hierarchical Creating new nodes, moving DAs [6] structure within the structure. Alter tags Add, delete or modify tags on DAs. [21] Hierarchical Browsing Tagged-based Browsing Matches existing mental model for desktops. E.g. Windows Explorer, Mac OS X Finder. Matches existing mental model for many applications. E.g. Google Desktop Search on table Matches existing mental model for desktop search. [13] [21] [14] [37,39] Table 5. Summary of Actions associated with DAs and their representatives Copy: to reproduce. The table distinguishes between two types of copy. The first type is copy DA. Making a copy of a DA duplicates the original underlying system object. The copy of the underlying object brings about a new digital artefact to the table. Each copy is independent of one another. A way of thinking about this is to think of the operation like that of a physical photocopier, where the original and copy are different products. The second type is copy linked DA. Making a copy of a linked DA makes a copy of the DA but any action performed on the original or copy is reflected on all other copied instances. An example is a Google 4 Actions that build on the a Desktop Mental Model 5 Literature 6 Literature that explicitly deals with the collaborative nature of a tabletop 14

16 Maps linked copy. When the original or copied content is interacted with, for example zoomed in, other linked instances follow suit in the operation. The implementation of copy in literature has seen various forms. For example Apted [7] uses a special photocopier object to create copies whilst Morris [36] has used a collaborative pull-apart team gesture for copying. Hide: to conceal from sight. The ability to hide a digital artefact needs to be considered carefully at a tabletop. A tabletop can have a number of digital artefacts and the ability to hide any artefact allows for the management of clutter and user focus for the remaining artefacts present. Delete: to erase or remove. Delete needs careful design for tabletops, especially if it is irreversible. Several approaches explored include: the necessity for people to agree on delete [36], the placement of a red cross (or similar icon) at the top of an artefact [34] and the use of a special object to denote deletion such as a black hole [7]. Transformation of a digital artefact representative to a digital artefact On a tabletop a digital artefact may be represented directly exposing the underlying artefact or it drawn through a representative. A representative is an item that acts as a placeholder for the actual artefact. On a tabletop this is commonly seen with icons being a compact representation of the real artefact. Upon manipulation of the icon, such as enlarging the representative is transformed into the artefact itself. Another example is that of opening an application. Often a label will take the place of the artefact and upon selection transforms into the program. Organization of digital artefacts Organization covers the last three parts of structure, browsing and searching. Digital artefacts in a system will often need to be co-ordinated into a unit or structure. This means that artefacts may be placed together in some way. A few methods that have been investigated include piling and grouping. Piling is explored by Agarawala [2] where artefacts are overlaid on top of each other similar to a stack of magazines in a Doctors office. Grouping is explored by Scott [56] and Trent [6] who look at storage bins for artefact aggregation. Similar work is explored by Pinelle [44] who utilizes flat trays to place artefacts together. In a tabletop system the structure of artefacts is not normally revealed to users. From the users perspective they see digital artefacts (or their representatives) on an interface and interact with them. An interface can expose its underlying structure (such as through visualization) but is under no obligation to do so. This means primitives for creating new nodes and manipulating artefacts are not tied down to any one particular structure. A tabletop gives rise to an expectation to browse information. This requires an ability to view available data sets and navigate through context. Browsing can take many forms on a tabletop. Most common on the desktop is hierarchi- 15

17 cal browsing, where artefacts are linked together in a tree like structure. This model has been explored on the tabletop by Collins [13]. Another mechanism is tag based browsing. This is where artefacts are given meta data describing characteristics that feed into actions to retrieve relevant content from stores. An example is Hartmann s [21] work on meta data group tagging. Similarly Collin s [14] work explores artefact retrieval through associative access. This type of access locates similar content from both the tabletop and devices attached to the system such as mobile phones. With the collaborative nature of a tabletop, new ways to locate content are required. In the literature, work by Morris [37] and Paepcke [39] have looked at collaborative search, comprised of boolean operators from different individuals to form search queries. Actions associated with personal and shared spaces Actions associated with spaces are explored in Table 6. It is important to understand the difference between types of spaces. Some actions are personal, meaning that the action is available to only that user whilst other actions are group based, being available to many or even needing many people to complete the action. Table 6 is split into three parts. The first focuses on individuality, the second on collocated group space and last on remote collaboration. Actions Brief Description Literature Individual What actions are specific to a user. [24] Private action on Make use of private device to perform private [55,60] personal mobile actions at table. device Group Coordination Support Remote Collaboration What actions are applicable to a set of users [58] together. What is required to help people out. [10,59,57,58] What is required for supported where participants are not collocated. Table 6. Summary of Actions associated with spaces [10,65] Individuality A user working on a tabletop may have operations that are specific to just that user. For example viewing a users private data set, or acting upon their digital artefacts or stores. While a tabletop promotes collaboration, this should not exclude operations that are private to individuals. Besides individual actions specific to a user, a user may bring to the system his or her own device (be it physical or wireless). This allows private actions to be performed on the device which can then be relayed to the tabletop. An example taking emergence is the 16

18 use of mobile phones to enter (or view) sensitive data [60] which is otherwise cumbersome at a public tabletop interface. Collocated Group Space There are often actions on a tabletop that are applicable to a set of users. These refer to how the space (or surface) of the tabletop is shared between participants. This requires an understanding of how a tabletop can enable many different types of actions to be performed concurrently. This has resulted in literature exploring how artefacts can be shared in tabletop environments. Seminal work is by Scott [58] who examines issues of territoriality on the tabletop. Remote Collaboration Remote Collaboration allows a tabletop is extend past a collocated space. This is where individuals are able to work on their own tabletop but mechanisms exist that promote collaboration where neither participant is physically next to each other. By using cameras and other technology, this research has been the focus of many groups. For example, the work by Tuddenham [64] examines high resolution tabletop prototyping for mixed-presence tabletop applications. This is achieved by superimposing shadows of remote participants onto local tables. Similarly work on remote collaboration is explored by Brave [10], Tang [63] and Wilson [68]. Actions to move DAs and groups of DAs within the digital ecosystem The ability for artefacts on a tabletop to move between devices in a digital ecology increases the usefulness of tabletop systems. Table 7 looks at these actions in three parts: export, import and synchronization. Actions Brief Description Literature Export DA Transfer DA(s) to other parts of digital ecosystem Import DA Synchronise DA Transfer DA(s) from other parts of digital ecosystem Ensure DA(s) kept consistent across other parts of digital ecosystem Table 7. Summary of Actions related to digital ecosystem [13,34] Export of Digital Artefacts A tabletop can be used as the medium upon which artefacts are discussed. Upon discussion it may be important to the user (or users) to take the content they interacted with, with them. Physically moving a tabletop is not practical due to size. Allowing artefacts to be transferred to other devices such as laptops, tablets and phones or ed to a location allows the experience that was had at the tabletop to travel with the user. So far, there has been minimal research 17

19 in the area of exporting digital artefacts, although some groups have started to look into the use of mobile phones in conjunction with a tabletop. Import of Digital Artefacts Bringing content to the tabletop removes the restriction of being limited to what is locally packaged with the tabletop. Importing has been visited in the work of Microsoft Surface [34], where external devices can be accessed through Bluetooth technology to retrieve the digital stores of the device for display on the table. The work of importing content has also been visited by Collins [14], who examined the release of files from a mobile phone onto the tabletop. Collin s work allowed the user to specify permissions, controlling which files were and were not allowed to be displayed on the tabletop. Synchronization Synchronization allows digital artefacts to be kept consistent across all other parts of a digital ecosystem. Synchronization is useful as changes reflected at one site are propagated to another. This action has yet to be the topic of much research. However, with products like Dropbox [16], which synchronizes files at different sites gaining popularity it may not be long until this area is explored. Actions on contexts A context is the surroundings, circumstances, environment, background, or settings which determine, specify, or clarify the meaning of an event. 7. Table 8 represents actions on contexts through the concept of navigation, saving and loading. Actions Brief Description Literature Navigate contexts Switch between work spaces. [24] Save context Ability to save a state for later use. It needs to be [24,34] saved in such a way that the system can understand and search over. Load context Ability to load a context from the past. There needs to be flexibility to find a context based on parameters. For example at the most basic: time, then if supported: users involved, meeting meta-data, etc. [24] Table 8. Summary of Actions on contexts

20 Navigate Contexts Navigation means to move between contexts. This refers to switching between different environments that are set up with tasks. In literature this is best described with the use of work spaces. A work space is an area where actions take place. Being able to navigate a work space, means to change where an action is invoked. For example, there may be a workspace for formal discussion and one for adhoc brainstorming. Each represents a different area. Hunter s work [24] has shown that it is feasible to have work spaces on a tabletop and he provides a simple mechanism for navigation. Save Context Saving preserves something in time. Saving a context captures the intent and purpose of the actions at that point. This is useful, for example when a number of people meet at a tabletop to have a discussion. The results of the discussion can be recorded for later analysis. In the literature, Hunter s work [24] allows for the state of the tabletop to be saved at any point in time, creating a historical log of operations. Other work includes the Microsoft Surface [34] which provides a mechanism entitled session management. This is an API designers and programmers can use to specify what an application can do in order to save a context in an application. Load Context Loading activates past events. This is useful when you want to restart from a given point in the past. Similarly to save, Hunter s work [24] allows for the loading of a context, by giving the user the ability to select any point in history to resume from. This is important because it allows you to begin your work without needing to go through a series of actions to get to the point where you left off. While many tabletop systems have not featured the actions of save and load, these actions are staples in the desktop world. With time, these actions are set to emerge, enabling a greater use of tabletops. Actions on Settings The last part deals with settings. Settings define the parameters that can be modified in a system. The flexibility of settings allows different users in a system, fileset choice and how artefacts can react to one another. The general area of tabletop settings has seen minimal work. There is a recent paper by Ackad [1] who looks at exploring this aspect of the tabletop. He focuses on application level configuration management. 19