BY CONTIGUOUS GESTURES

Transcription

1 GEDIT: A TEST BED FOR EDITIN G BY CONTIGUOUS GESTURES GORDON KURTENBAC H BILL BUXTON INTRODUCTIO N GEdit is a prototype graphical editor that permits you to create and manipulate three simple types of objects usin g shorthand and proof-reader's type gestures. The three types of objects that can be manipulated are : squares, circles and triangles. Using hand-drawn symbols, the user can add, delete, move and copy these objects. Objects can be manipulated individually or in groups. Groups are specified with hand-drawn circling symbols. GEdit provides a toy world that serves to demonstrate a number of concepts : The use of proof-reader's like symbols (rather than the more common alphanumeric character recognition) ; The use of where a symbol is drawn as well as what symbol is drawn to determine intent; The use of compound symbols, that is, symbols (such as move) that have more than one "token " embedded in a single continuous line. For example, in contrast to a character recognizer where what is recognized is at the level of the letter "A," the proof-reader's symbol embeds three different pieces of information : the command (move), what is to be moved (direct object), and where it is to be moved to (indirect object). 1 GEdit is available from the authors at no charge. To receive a copy of the Macintosh version, just send the authors a blank formatted floppy dis k and a mailing label containing your return address. Contact the authors for more information if you are interested in obtaining the Sun version. GEdit runs on any Apple Macintosh computer. A version for the Sun (Suntools and X11) also exists. The program i s available from the authors as shareware. The hope is that it will provide a useful educational tool for those interested in character and gesture recognition. The intent of thi s document is to introduce the program and to serve a s documentation for those using it. l THE TOY WORL D GEdit permits the creation and manipulation of three type s of objects (squares, circles and triangles) in the context of a graphical layout program. It builds upon two earlier studie s that investigated using similar symbols and object s (Buxton, 1982, and Buxton, Fiume, Hill, Lee & Woo, 1983). This particular toy world was chosen because it is simpl e enough to be tractable, yet complex enough to capture a critical mass of real-world problems (such as one encounters in a CAD or graphical layout program). BASIC OPERATION OF GEDIT One interacts with GEdit using a number of simpl e gestures. These are articulated using a pointing device, such as a mouse. Commands are entered by moving th e pointing device while holding down the associated button or switch. Most commands leave an "ink trail" along th e path of motion. SIGCHI Bulletin April Volume 23, Number 2

2 Adding Objects Objects are created using a form of shorthand notation. Each of the three objects is created using a specific symbol. These are illustrated in Figure 1 : Copying Individual Objects There is no mechanism for copying individual objects. The rationale for this is that it is easier to create a new objec t than it is to copy an existing one. SCOPE : IMMEDIATE AND COLLECTIV E GEdit permits operations to be performed upon object s individually or in groups. The process of specifying the object(s) to be operated upon is known as specifying th e scope of the operator. When the scope is a single object, we refer to it as immediate scope. All of the examples thus far have been immediate scope. Figure 1 : Shorthand Symbols for Adding Object s Three objects can be defined: a square, circle and triangle. These are illustrated using bold lines. Shorthand symbols are used to define which type of object is to be created, and where it is to be placed. These are illustrated using thin lines with arrowheads. Object typ e is defined by the shape of the shorthand symbol. The new object is centred on the starting point of th e defining symbol. The object being defined is centered on the starting point from which the shorthand symbol is drawn. Thus, the symbol defines both shape and position of the new object. Deleting an Individual Object Another shorthand symbol can be used to delete individua l objects. This is done by drawing through the object with a single horizontal stroke, as illustrated in Figure 2. When the scope includes more than one object, we refer to it as collective scope. In GEdit, the mechanism for specifying collective scope is circling. Scoping mechanisms have a large impact on the effectiveness of a system (Buxton, Patel, Reeves & Baecker, 1981). GEdit provides an opportunity to explore some not so common aspects of scope specification. These are illustrated in the examples that follow. COLLECTIVE OPERATIONS IN GEDI T Deleting a Group of Objects A group of objects can be deleted by circling them, an d extending the encircling line so that it terminates within th e circle. This is illustrated in Figure 3. Figure 2: Deleting an Individual Object Individual objects can be deleted by drawing a single horizontal stroke through them. The direction of th e stroke does not matter. The direction of the stroke does not matter. Moving an Individual Objec t Single objects are moved in GEdit by dragging. The technique used is the same as how icons are moved in mos t direct manipulation systems. For example, using a mouse as the pointing device, point at the object to be moved, depress the mouse button, drag the object to the desire d position by moving the mouse, then anchor it in that position by releasing the mouse button. Figure 3 : Deleting a Group of Object s A group of objects is deleted by circling them, an d extending the encircling line within the circle. SIGCHI Bulletin April Volume 23, Number 2

3 Moving a Group of Objects The proof-reader's symbol for move is used to move a group of objects. The objects to be moved are encircled, and the encircling line extended to the point to which th e objects are to be moved. This is illustrated in Figure 4. Figure 5 : Copying a Group of Object s A group of objects is copied by circling them, and extending the encircling line to the new location. and drawing a "C" (for copy) gesture. Figure 4: Moving a Group of Object s A group of objects is moved by circling them, an d extending the encircling line to the new location. The move command confronts us with two important points worthy of discussion. Conflict between symbols : Note that in certain cases, there is a conflict between the move and the delet e symbols. In particular, a conflict occurs when th e amount of movement is slight, and would result in th e endpoint of the defining line falling within the circle. The example illustrates the fact that gestures hav e impact on one another and must be considered in th e larger context of an application. a problem. The reason is that there is that the dragged objects have a "weight" that would make the execution o f the "C" gesture inappropriate while dragging. Deferred Operation The move and copy symbols share the property that th e entire command (verb, direct object and indirect object) ca n be articulated using a single uninterrupted line. Sometimes, however, this is not desired. GEdit provides an alternativ e mechanism, whereby the scope and operation can b e specified in two steps. In such cases, the scope is first specified using a circlin g gesture, as illustrated in Figure 6. Lack of dragging : Unlike moving individual objects (immediate scope), the group of objects is not dragged to the new position., As a result, there is no possibility to preview the results of the operation before commitin g to the move. When working with graphical objects, thi s is a problem, since it hampers the precise placement of moved objects. Note however, that if what was being moved was running text in a word processor, this woul d not be a problem. In this case, we don't want the indicated text to be moved until the insertion point i s indicated. The example illustrates how the feedback of an action must depend on the type of data being acted upon as well as what operator is being used. 0 Copying a Group of Objects The command for copying groups of objects is similar t o that move operator. To copy, one adds a small "C" symbol to the end of the gesture. This is illustrated in Figure 5. The gesture works, but suffers from the same problems a s the move gesture, namely, conflict with the delet e command and lack of dragging, or preview. The issue of dragging is interesting, in that even if it were implemented, this particular approach to specifying "copy" would have Figure 6 : Specifying Scope Alone. Scope can be specified without specifying an operator. In this case, by way of feed-back, the system provides a graphical "handle" in the way of a small square at th e end-point of the circle. The delete, move, or copy gestures can then be articulated as long as the command portion is started in the box handle. SIGCHI Bulletin April Volume 23, Number 2

4 In this case, where no command accompanies th e specification of the scope, feedback is provided by way of a small box appearing at the end of the encircling line. Thi s box is like a "handle" on the scope. Having thus specifie d the scope, one can then invoke the delete, move or cop y operator by drawing the command part of the gesture, starting within this "handle." Scope with Exception s One of the prime shortcomings of many scope specificatio n techniques, such as circling or "drag through," is that the y are all encompassing. How does one handle cases where one doesn't want to operate on all of the encircled objects? The prime reason that GEdit included the independen t specification of scope, as discussed in the previous section, was to provide a means whereby some objects within th e specified region could be excluded from being operate d upon. This is made possible by enabling circles of exclusion a s well as inclusion. This is illustrated in Figure 7. articulate the user's intent. While GEdit is just a to y program, it brings forward a number of significant points. Some of these have been discussed above. A fe w additional points are discussed briefly in the remainder of this section. Choice of symbol: Unlike menu-based systems, this clas s of interface is not self-revealing. It is not always obviou s what commands are possible or how to invoke them. A s the repertoire of commands grows, so does the problem of remembering the symbols that invoke them. Graphica l mnemonics can help the situation through the use of symbols that have some graphical correspondence with what they represent. In GEdit, for example, this is the case with the triangl e symbol. On the other hand, this is not the case with th e symbols for the square and circle, which were chosen fo r the speed with which they could be input. The program confronts us with the potential conflict between optimizin g two aspects of human performance : memory and execution time. Input device : The majority of studies of input technologie s have focused on a limited number of tasks, especiall y pursuit tracking and target acquisition. These studies d o not provide us with much information about how well various technologies an support the "line-drive" type o f interface demonstrated in GEdit. One advantage to the Macintosh version of the program is that it is very easy to change input devices for the purpos e of comparison. Perhaps the most telling transaction is how consistently one can input the triangle symbol. Figure 7: Specifying noninclusive scope. Circles of both inclusion and exclusion can be specified. The outermost circle is always inclusion. Circles withi n circles toggle between exclusion and inclusion. In th e example, all objects are selected except for the triangles. This technique is useful in cases where the objects to b e operated have a high degree of spatial coherence, but where there are other objects also in the same area. In th e example illustrated in Figure 7, for example, the intentio n is to operate on all objects except for the triangles. Deleting the intended objects could, for example, b e accomplished by drawing a stroke starting from th e outermost circle's "handle" and terminating anywher e within. DISCUSSIO N GEdit is an example of what could be called a "line drive " interface. That is, an interface in which lines are used to The recognizer is intentionally not very tolerant in what it will accept for this command. What users will quickl y discover is that the mouse performs far worst than a stylus or the finger. This example makes the point that if th e input device has been decided a priori, then the symbols must be designed to match the inherent strengths and weaknesses of that device. (Note, for example, that there i s no problem articulating the delete, square or circle symbol s with a mouse.) Disjoint scope : Once a mechanism is provided to specif y scope incrementally, a logical extension is to enabl e disjoint objects to be selected. An example of this is illustrated in Figure 8. In the example, all of the triangles are selected through the use of two disjoint circles. Having come this far, however, we are confronted with a design dilemma. We hav e established a convention that the square box at the endpoin t of the scope-defining line is a "handle" from which an y SIGCHI Bulletin April Volume 23, Number 2

5 subsequent command must be initiated if it is to affect th e encircled objects. However, in this case, there are two suc h "outer circles." The dilemma is, if both are defined, should an operation on one simultaneously affect the other? In GEdit, the answe r is no. Let us assume that the operation to be performed is move. Using Figure 8 as an example, we would first mov e the triangle(s) in one circle, and then the other. The circl e defining a particular group disappears after the operation o n that group is finished. Other disjoint circles remain until acted upon. While it works, this approach means that it takes two step s to do what might otherwise be done in one, and it i s difficult to preserve the spatial relationship among the tw o sets of triangles during the move. specification, as in Fig. 8, scope persists until the object s are operated upon. This hints at persistence. However, the range of operations in the program is not rich enough t o really push on the issue. For example, the issue would be far more germane if we could select a group of objects, move them, add a few more, then scale or rotate all of them. The transaction is representative of a number of real-worl d tasks that are currently poorly supported. It is a shortcoming of GEdit that it is not rich enough to allow us to fully explore this issue. Nevertheless, even if it provides a hint that the issue needs attention, it has served som e useful purpose. CONCLUSION S GEdit is a seemingly simple program that affords us th e ability to gain valuable experience in a relatively uncharte d area of HCI. The value of GEdit is not so much in how it has implemented its solution, as in its bringing the issue t o the fore. We hope that it makes some contribution to th e general literacy about "line drive" interfaces, and helps demonstrate the value of such critical-mass toy programs. ACKNOWLEDGEMENTS Figure 8 : Disjoint scope specification. Ideally, disjoint clusters of objects can be selected as th e scope of an operator. In this case, multiple circles are used. In contrast to Figure 7, in this case, only the triangles are selected. Temporal pervasiveness of scope : The previous exampl e serves to illustrate a further point. With direct manipulation interfaces, there is no systematic way in which scope can remain in effect from operation t o operation, especially if interspaced with any othe r transactions (such as window manager commands) not acting on the selected objects. This is an unsolved problem. Without imposing a burde n larger than the problem that we are trying to solve, how can we easily indicate when we want the scope to persist from transaction to transaction? Using disjoint scope The research described in this paper has been undertake with the generous support of the Natural Sciences an d Engineering Research Council of Canada, Xerox PARC, Apple Computer and Digital Equipment Corp. Thi s support of our laboratory and research is gratefull y acknowledged. REFERENCE S Buxton, W. (1982). An informal study of selectionpositioning tasks. Proceedings of Graphics Interface `82, 8th Conference of the Canadian Man-Compute r Communications Society, Toronto, May, 1982, Buxton, W., Fiume, E., Hill, R., Lee, A. & Woo, C. (1983). Continuous Hand-Gesture Driven Input. Proceedings of Graphics Interface `83, Edmonton, Ma y 1983, Buxton, W., Patel, S., Reeves, W., & Baecker, R. (1981). Scope in Interactive Score Editors. Computer Music Journal 5 (3), SIGCHI Bulletin April Volume 23, Number 2