Simple() Select( Text ) Select( Line ) Select(type) ClickSel(type) AreaSel(type)

Transcription

1 Copyright 1994 IEEE. Published in the Proceedings of VL '94, 4-7 October 1994 at St. Louis, Missouri, USA Interaction in Really Graphical User Interfaces G. Viehstaedt M. Minas Lehrstuhl fur Programmiersprachen Universitat Erlangen-Nurnberg Martensstr. 3, Erlangen, Germany Abstract Employing diagrams in the UI causes problems that don't exist in so called Graphical User Interfaces. We are implementing a tool for generating editors for (a certain class of) diagrams from a specication. Diagram Editors are needed inreally graphical UIs and interactive visual language environments. This paper addresses three aspects of the specication of diagram editors. After highlighting diculties in user interaction with diagrams event automata are suggested as a model for specication of dialogs. Furthermore, it is discussed how objects selected by the user can automatically be assembled into semantically meaningful groups. These are needed for more complex editing operations. Aneasy and versatile way of specifying layout for diagram editors by constraints has been introduced in [1] and is generalized here. 1 Introduction So called Graphical User Interfaces (GUIs) are actually not graphical, but based on widgets. We share the opinion voiced in a recent CACM issue on GUIs: What the market considers a GUI is little more than a gloried menu system, having no graphics. This leaves the graphical representation of the application domain as an exercise for the developer, ::: [2]. Diagrams in addition to widgets and interaction with diagrams make `G'UIs more graphical, but also give raise to problems that don't exist in widget based UIs. Using a general purpose drawing editor isn't an adequate way for dealing with diagrams. Generally, diagrams are needed as component of a GUI or interactive visual language environment and have semantics relevant to the application. Thus, we can't permit any composition of pimitive graphical objects, but have to keep the diagram within some class of diagrams. A diagram editor is a syntax-directed editor tailored to a certain class of diagrams, e.g., Nassi-Shneiderman diagrams (NSDs). Layout and complex editing operations can be supported by a diagram editor, which is hardly possible for a general drawing program. Similar to the Synthesizer Generator [3] for syntaxdirected text editors, diagram editors should be built from a specication using a tool. Currently, we are implementing the Diagram Editor Generator DEG. An editor's specication includes the denition of the diagram class by means of a constraint hypergraph grammar (CHG) [1]. This kind of grammar permits easy specication of layout for graphic objects. In the rst section, we focus on specifying dialog control for editing diagrams. After highlighting the specic problems, event automata are introduced. Abstraction for events is realized since transitions can be caused by user-dened, high level events. Event automata are based on nite state machines and therefore have an intuitive visual representation. Parallel composition of automata supports multi-threaded dialogs. The next section deals with the problem of assembling a set of selected objects into groups. This is done automatically, taking into account the specication of a group's structure. Such a specication is based on the grammar given for a diagram class. CHGs and the incremental algorithm for constraint evaluation presented in [1] make possible integration of automatic diagram layout with user dened modications. Constraints consisting of linear equations and inequalities specify layout. We show how this concept can be generalized to include nonlinear constraints. 2 Event Automata 2.1 User interaction with diagrams Widgets as available in the most prevalent toolkits can be parameterized, their behavior, however, is rather xed. The widget is responsible for sensing primitive events such as pressing a mouse button. The user of a widget doesn't have to deal with these primitive

2 "text"(ul,ll,ur,lr) = // ul: upper left node { Text < > (ul,ll,ur,lr); Handle < arity = 4 > (ul,ll,ur,lr); RectangleReceptor < receptor_type = "Text" > (ul,ll,ur,lr); }; "line"(n1,n2) = { Line (n1,n2); Handle < arity = 2 > (n1,n2); LineReceptor < receptor_type = "Line", tolerance = 5 > (n1,n2); }; Figure 1: Example diagram with selected line for section 2.1, resp. NSD go() referenced in section 2.3 events, but receives a notication, if a higher level event (e.g., selection of a menu item) has happened. Usually, this is done by invoking a callback function that has been registered by the user. The behavior of a widget is determined by the callback functions that can be registered. Toolkits provide for composition of widgets into more complex ones, e.g., a couple of menu items make upa menu. Since component widgets don't overlap, it is clear where primitiveevents go. Even if there is an overlap of widgets during interaction (consider pulldown menus), the behavior is well dened since only the topmost widget receives the events. The situation is entirely dierent for diagrams. First, we do not have xed behavior. Instead, it has to be specied how primitiveevents are composed into higher level events. Second, diagram objects often overlap. Even if output objects don't, this can be the case for areas sensitive to input. Consider the NSD example in Fig. 1, which is a snapshot from our system. A diagram is composed of lines and text blocks. A line can be selected by clicking. Diagram objects consist of primitive components for output and handles for indicating selection. Areas sensitive to input, subsequently called receptors, are needed as well. In DEG, diagram objects are terminal symbols (TSs) in the grammar for NSDs and in the simplest case dened as follows: TSs are hyperedges connected to a number of nodes, e.g., text is connected to the four corners of a text block. Each node (ul, :::) represents a point in the plane and is used for specifying layout (see [1] for details). tolerance indicates the width of a line receptor in number of pixels as needed for clicking. This width causes overlap with text receptors. Abstraction is a major concern for specication of dialogs. Selection of a line, e.g., can be seen as a high level event. In contrast to low level events such as pressing a mouse button, visual feedback isgiven. Thus, high level events can have a dual nature of being actions as well. 2.2 Models for interaction The most prevalent models for interaction are state transition systems and event systems. In order to elucidate their inherent characteristics, we will rst compare the basics of these models. Then Statecharts [4] and the multiagent model, which are extensions of state transition resp. event systems [5, 6] are addressed. State transition systems are based on the well-known nite state machines (FSM). An event system consists of a collection of rules having the form `if condition then action'. Rules are red, if the system's current state satises the condition part. In the action part, the system's state can be changed and/or new events can be sent, i.e., high level events can be generated. The strength of the event model is that there is no sequencing inherent to the rules. This is essential for widget-based UIs, where some user actions can often be performed in any order. Finite state machines, in contrast, exhibit sequential behavior. The event model's strength is also its weakness. Firing a rule has global consequences since its action can turn any other rule's condition part to true. Due to the lack of explicit transfer of control it is dicult to track down the system's behavior, esp. if the condition part of more than one rule is fullled. This case can easily lead to nondeterminism, if two actions modify the same global data item. Generally, the execution engine

3 Simple() Edit() Select(type) Select( Text ) ClickSel(type) AreaSel(type) Select( Line ) Move() Figure 2: Simple automata for describing selection of items. Text or Line objects can be selected in Simple(). Select() permits two ways for selection. of an event system will enforce some order, but the user doesn't know if such unwanted conicts occur. In order to nd out about these conicts, an analysis of the entire system is necessary. If we want to tell whether a FSM is deterministic or not, a local analysis (i.e., for each state) is sucient. Except from Harel's Statecharts, there have been several earlier suggestions for extending FSMs [7, 8]. We consider Statecharts here since it has gained some popularity and is the rst to incorporate parallelism. Statecharts is a very powerful model where states can be arranged in a hierarchy and actions permit generating user-dened, high level events. The multiagent model adds encapsulation to the event model by using techniques from object-oriented programming. This includes hierarchical composition and a local state that can serve for limiting the state changes caused by anevent toawell-dened part of the system. If events coming from the level below need to arrive in a particular sequence, the agent has to keep track of this using the local state. I.e. some implementation of a FSM is needed inside the agent. Obviously, the extensions diminish the dierences between the basic models. A major point, however, is that Statecharts is a visual formalism, whereas the multiagent approach is more like a special purpose programming language. (The model-view-controller model [9] is an example.) In our opinion, a visual formalism is preferable for the domain of user interaction with diagrams. Thus, we will now highlight what makes using Statecharts troublesome in this application. Consider a very simple example for interaction in an NSD. Suppose updates to the layout are made by selecting a line that can then be moved. Also, we may require that a text be selected prior to editing. The most natural solution is to have a Select() that can be specialized to the type of receptor (input area) we want to select and a state from which both Select() edges start, as in Simple() in Fig. 2. Dierent target nodes make it easy and visual to allow for operations that depend on the item selected. In Statecharts, only transitions labeled `event(condition)' are permitted, but an event can be a high level event generated by another automaton running in parallel to Simple(). Thus, we have to revert to something like `Select(x = "Line")' with a global variable x or to two dierentevents SelectLine and SelectText to accommodate both cases. The main problem, however, is that due to the overlap of receptors both events may be generated. This results in a nondeterministic system and is an erroneous specication from the user's point of view [4]. To avoid nondeterminism, both Select()s can be combined into a single automaton or a global variable might be used for communication between both automata. In any case, the solution doesn't leave the two Select()s independent of each other. A similar problem shows up on the next level of our example in Fig. 2, where ClickSel() and AreaSel() are possible to select an item by clicking resp. indicating a selection area. Assume that the same mouse button is used for both kinds and area selection is done in the usual way with a selection box being displayed. This time the problem isn't due to overlap of receptors, but results from the event sequence ButtonDown, ButtonUp for ClickSel() being a subsequence of that for AreaSel(). In Statecharts, an automaton A stays in its current state, if an event doesn't t any arc emanating from this state. One reason for this solution is that events intended for an automaton A 0 running in parallel to A have no eect and that A and A 0 don't have to be modied only because they run in parallel. Actually, the same notion of staying in the current state exists in the event model, as, e.g., in [10]. Thus, both ClickSel() and AreaSel() may occur. Feedback as given by AreaSel() will result in confusion if more than one action is involved. Again, this diculty has to be solved in Statecharts by giving up independence of ClickSel() and AreaSel() at an early stage, i.e., before both are used in Select(). It is easy to imagine that this lack of independence is undesirable and esp. complicates extensions or changes to dialog specications. One of the reasons for these dif- culties is that events in Statecharts abandon some of the advantages of FSMs 1. In particular, a global analysis of the system has to be performed to ensure deterministic behavior, since a high level event can come from any other automaton running in parallel. The other reason is that for diagrams a limited form of nondeterminism is highly desirable, if we want to allow for abstraction. This is not provided by Statecharts. 1 Furthermore, Harel mentions in [4] that events make denition of the semantics very dicult.

4 Simple() Edit() Deselect() Deselect() Move() Select( Text ) Select( Line ) Select(type) ClickSel(type) AreaSel(type) <otherwise> Figure 3: Event automata for selection as explained in Fig. 2. The little arrowheads on the borderline of states indicate a priority for outgoing arcs. 2.3 The event automata model We try to avoid the aforementioned problems by temporarily permitting nondeterminism. If necessary, nondeterminism is resolved by the execution engine in a lucid way. An event automaton can be a simple or parallel automaton. A parallel automaton is composed of simple automata and will be addressed later. The example given above can be written down as event automata Simple() and Select() in an intuitiveway, see Fig. 3. Edges are labeled with conditions that include primitive events or with subautomata as in the example. A sequence of events leading to successful execution of a subautomaton can also be seen as high level event. Initial states are indicated by grey ll pattern, - nal states by double circles. There can be more than one nal state or none 2, as in Simple(). When, e.g., Simple() is started, it is tried to reach another state from the initial state. Thus, all subautomata on edges emanating from the initial state are started. Low level events coming from the window system are processed by the active subautomata (those that have been started, but have not yet terminated execution). Simple() enters a new state, if one of these subautomata reaches a nal state. Due to overlap of receptors, this might happen for more than one at the same time. To ensure deterministic behavior, a priority is assigned to all edges starting from some node. Priorityis indicated by the small arrow head attached to the starting point ofthe edge with highest priority. This gives Select("Line") priorityover subsequent arcs. In the second part of the example, AreaSel() includes transitions producing output for feedback. If a transition's action (not shown here) includes an output operation, other branches such as ClickSel() in our case are terminated. This is another way how nondeter- 2 Thus, edges are indeed hyperedges with a single source and a number of target nodes. Figure 4: Subdiagram gotrytrans of NSD go() in Fig. 1. The handles in this snapshot are displayed for selected items and serve as an example in section 3. Figure 5: Subdiagram gosetstate of NSD go() in Fig. 1. Some details of the actual algorithm have been omitted. minism is resolved. In Simple() we obviously assume that Select() nds an object of the desired type. This means that ClickSel() (and AreaSel() as well) must not reach a nal state, unless such an object is found. To prevent subautomata from `getting stuck' after a couple of events, they can signal unsuccessful termination by a transition into a give-up state. This is a (non-nal) state without emanating edges. If all subautomata are unsuccessfully terminated and there is an <otherwise>transition, this is taken. In our example, not nding an appropriate object in ClickSel() or AreaSel() leads to unsuccessful termination of Select(). Since there isn't any <otherwise> in Simple(), this automaton stays in its current state (the initial state) and the next primitive events arriving may cause a transition. The NSDs in Figs. 1, 4, and 5 give an idea of how the execution engine for simple event automata works. When a low level event is delivered from the window

5 system, it is propagated downwards in the hierarchy of currently running automata by go(event) (Fig. 1). A C++-like notation is used, thus A.x refers to a member function or instance variable x of A. The value of status is returned by go() and can either be <NoStepDone> for no progress in A or one of A's subautomata, or <StepDone> for some progress, or <GiveUp> if A has reached a give-up state, or one of A's nal states if A has entered such a state. These return values are relevant to deciding on the level above which of the nondeterministic branches shall be followed in the next step. A value of new state dierent from undefined indicates that a transition in A has been completed. In the for-loop in Fig. 1 the transitions starting from A's current state are tried in the sequence of their priority until the rst has completed a transition in A or all have been tried. This way, execution is made deterministic. If a new state has been reached in A, A's current state and the return value status are updated according to subdiagram gosetstate. If the new state isn't a nal state, the until does not yet nish, and further transitions are tried. (Due to having set current event to undefined, only transitions with conditions that don't require an event can then be made.) In subdiagram gotrytrans (Fig. 4) it is tried to reach a target node of transition t. t can be a simple or a complex transition. In the former case, t consists of a condition (which can include a primitiveevent) and an action. A complex condition references some subautomaton S. If an additional condition is given, this has to be true only when S is started. The return value received from S.go(event) may dene new values for A's return value status and for new state. In gosetstate (Fig. 5) A's current state A.state and status may be set. If a new state has not yet been reached in A since there are still some subautomata running, it is decided here which of the nondeterministic branches shall be followed in the next step. These are kept as the list of active transitions, which is determined by the return values from the subautomata. The intention is to follow only those branches that `t' the sequence of primitiveevents received so far and terminate the others. (These branches are also redened by actions that cause output, as explained before.) In case all subautomata have unsuccessfully terminated execution, A's state depends on the existence of an <otherwise>transition. We will now briey explain parallel automata. A parallel automaton A consists of a list of constituents that are simple automata, a list of nal states, and a termination table. By means of this table, termination of A is dened with respect to (successful) termination of A's constituents. If all of them terminate, this ensues ter- Buttons() Quit() OtherButtons() Figure 6: Basic automaton for the buttons on the lhs of Fig. 1 with subautomata Quit() for quitting and OtherButtons() for other buttons actions. mination of A. On the other hand, A can terminate as soon as at least one of its constituents terminates. This way interrupt like eects of an event can be specied, as is possible in Statecharts. go(event) for a parallel automaton simply works by mapping to the constituents that did not yet terminate and by checking the termination table each time one of the constituents nishes execution. Suppose, interaction for the buttons on the left hand side of Fig. 1 is specied by automaton Buttons(), depicted in Fig. 6. Then automaton Main() can be dened as parallel composition of Buttons() and Simple(). Main() will have a single nal state and its termination table denes that Main() nishes as soon as Buttons() terminates via Quit(), regardless of Simple()'s current state. This avoids a multitude of transitions for Quit() that would be needed without having parallel composition and user specied termination. 3 Groups When dealing with diagrams, everybody will expect to be able to deal with a set of (primitive) objects in some action. Groups address this problem. In a general purpose drawing editor the user can arrange objects hierarchically into groups. This is somewhat inconvenient since an explicit action is required and the result depends on the sequence of actions for selection and grouping. Furthermore, any objects can be put together into a group, i.e., a group isn't more than a set of unrelated objects. Complex editing operations, however, can require a special structure for a group. When, e.g. a sequence of statements in the NSD example is arranged into a group, we want to deal with it as a sequence and not as a set of unrelated statements. This is possible in the Synthesizer Generator [3], where a group is assembled automatically based on the objects selected by the user. The structure groups can have is limited to entire subtrees of the derivation tree and lists. For lists, the productions have to have a special form. User interaction is based on editing a number of buers in a cut-and-paste style. Just a single selec-

6 tion is possible in each buer, with the consequence that grouping is not independent of the order of select operations. This looks acceptable for text editors, but for diagram editors a more general concept is desirable. Attribute grammars lend themselves to a declarative specication of groups. Assembling selected objects into groups can be achieved by attribute evaluation in an order-independentway. This approachwould oer highest exibility, but has two major disadvantages. First, specication will get rather tedious since several aspects of a group (structure, eects of (de)select, visualization) have to be specied by attribution. Second, when using groups in transformations of the diagram, a group's shape (i.e., the way a group looks like in the derivation tree) is needed. For general attribution any shape can emerge which can't be handled in transformations. In DEG we adopted a model for groups that is based on attribute grammars and oers declarative and orderindependent grouping. The structure of a group is specied by agroup pattern consisting of a number of rules. Groups are dened automatically in a hierarchical way according to these rules. This results in restricted shapes which can be used in transformations without any trouble. Starting point for group denition are the terminal symbols (TSs) currently selected, mirrored by an attribute Sel for each TS. TSs form subgroups; TSs and subgroups form new, bigger subgroups. A subgroup denes a group, if it is not contained in another subgroup. Composition of groups is done bottom-up in the derivation tree that represents a diagram in DEG. Thus, a group is bound to some NTS N in the derivation tree and all members of the group can be found in the subtree with root N. The rules that caused denition of a group tell which TSs and subgroups are member of a (sub)group. A rule is associated with some production in the grammar given for the diagram class. There is a default group Sel extending selection to NTSs resp. indicating selection of an entire subtree. For the NSD example productions and group patterns for groups Sel and Seq (for a sequence of statements) are: NSD(n1,n2,n3,n4) ::= P1: Sts(n1,n2,n3,n4) "v_line"(n1,n2)... // more terminal symbols < group "Sel": 1; > {}. // (1) Sts(n1,n2,n3,n4) ::= // statement sequence P2: St(n1,n2,n3,n4) {} P3: St(n1,n5,n3,n6) Sts(n5,n2,n6,n4) "h_line"(n5,n6) < group "Sel": 1,2; > {}. St(n1,n2,n3,n4) ::= // statement P4: "text"(n1,n2,n3,n4) {... // Constraints for layout } Figure 7: Single group after further selection for the diagram in Fig. 4. The diagram has been clipped to the relevant part.... group Seq = { P2(Sel); // (2) P3(Sel,!Seq,?); P3(Sel, Seq,?); }; We won't go into the details, but try to explain the concept for group patterns. For group Sel a straightforward default rule for assembling bigger Sel-subgroups tells that there must be a Sel-subgroup for all symbols on a production's rhs in order to form a Sel-subgroup for the production's lhs. The default can be overridden as, e.g. in P1, where just the Sel-subgroup for the rst symbol on P1's rhs, Sts, is required (see (1)). The group pattern for Seq consists of three rules, see (2). The rst says that there is a Seq-subgroup for P2's lhs, if we haveasel-subgroup for St on the rhs. This is a pretty small sequence of statements with just one element. The same applies for the second rule, where a Sel-subgroup for the rst symbol on P3's rhs, St, must exist and there must not be Seq-subgroup for the second. In the third rule, a Sel- and a Seq-subgroup are combined into a bigger Seq-subgroup. 3 The eects of these specications will be illustrated by a tiny example. We start from Fig. 4, where we have twoseq-groups indicated by handles. The upper one consists of two consecutive statements, the lower one of a single statement. Suppose now that there is a `click' on the text between these groups and this causes a select() action for the text-ts. Then groups are automatically restructured according to the new selection. Here, both Seq-groups are combined into a single bigger one, resulting in the image depicted in Fig. 7. The result is independent of the order of selection. Groups dene temporary diagram objects that can be used for interaction like primitive objects (i.e., TSs). In particular, groups catch events as well as receptors of TSs. Primitive objects contained in a group are still accessible (in contrast to the primitive groups in drawing editors). The denition of groups thus doesn't cause any 3 Although the impression of a pure bottom-up procedure arises, this is actually not the case, but can't be addressed here.

7 restriction for interaction, but rather oers compound objects with a certain structure. Visualization of selected objects can be confusing, if many are selected and each is marked by handles. To avoid this, handles are displayed sparingly by not displaying handles for TSs that are member of some group. For groups, only nodes on the group's `border' (i.e., nodes connected to a TS that is member of the group as well as to a TS not in the group) carry handles. The system takes care of this, based on the group patterns dened. For a select() action issued for a TS it is furthermore possible to `propagate selection' such that a whole set of TSs can be selected due to a single primitive action. E.g. we could dene that a single `shift-click' on the header of an NSD selects the entire diagram. Selection in the Synthesizer Generator doesn't oer such exibility. 4 Layout Since layout of diagram objects frequently is a matter of personal taste, automatic layout should be integrated with user-dened modications. This is possible in an easy and versatile way using constraint hypergraph grammars [1]. A diagram's structure is dened by a grammar as illustrated in section 3, p.6, with TSs as in 2.1, p.2. Symbols are hyperedges, i.e., edges connected to a number of nodes. Each node stands for a point in the plane having an x- and y-attribute. Layout is specied by a set of constraints for these attributes. For userdened modications to the system's layout, inequality constraints are needed. They have been introduced in [1] together with an incremental algorithm for solving these constraints, but were restricted to linear equations and inequalities. Nonlinear constraints had to be avoided. Even for the NSD example in [1], however, nonlinear constraints make sense: the size needed for a text block should be large enough to show its text entirely. In terms of its width w and height h (in number of lines), this results in a nonlinear constraint. Strongly simplied, the constraint is w h c n, where c is acharacter's width and n is the number of characters. 4 Searching for tight bounds for w as well as h is a discrete optimization problem. However, the constraint can be written as h f h (w) with a nonlinear function f h. There are two possibilities how to deal with these constraints, presented in the following. 4 This constraint is nonlinear since w and h are unknown variables. Linear constraints have the form c1 x1 + :::+cnxn c0 with constants ci. 4.1 Avoiding nonlinear constraints The simplest possibilityisto not consider the nonlinear constraints during constraint solving. These constraints are rather restricted to a sharper linear constraint. Within the NSD example, this is the way used so far: each text block has a predened, constant width w 0. This implies a constraint w w 0. Each time when a text block is edited, f h (w 0 ) is evaluated and an appropriate height-constraint h f h (w 0 ) is inserted. Both constraints are obviously a linear restriction of h f h (w). Thus, each solution using these sharper linear constraints is a solution of the original constraint system. This procedure has the disadvantage, that a text block's width can never shrink below w 0 unless special measures are taken. Furthermore, a text block's height is never less than f h (w 0 ), even if its width is fairly greater than w 0. This may lead to text blocks that use only a small space fragment for text. To adjust the minimal width or height, a special method must be provided. Adjustments can be made by using this method in actions of an event automaton. These special methods, the inconvenience of using them, and the xed limits for width and height are a consequence of excluding nonlinear constraints during constraint system solving. The following idea includes nonlinear constraints in the solving process, however in a dierent way than the linear ones. 4.2 Partitioning the constraint system In the NSD example, each linear constraint restricts only x-attributes or only y-attributes, but never simultaneously x- and y-attributes. Only the nonlinear constraints combine x- and y-attributes. Thus, it is obvious to partition the whole constraint system (which, of course, depends on the current NSD) into an x- and a y-system, each consisting of only linear constraints. The nonlinear constraints link both systems, but don't belong to any of them. A solution of the whole constraint system, including the nonlinear constraints, can be found as follows: rst, the x-system is solved using the known algorithm. During this procedure, the nonlinear constraints are included as linear constraints projected to their x- attributes. As the second step, the y-system is solved. While doing so, the nonlinear constraints with their x- attributes xed are considered as linear y-constraints. This solving procedure works due to two reasons: The nonlinear constraints have linear projections in their x- and y-attributes. In the rst step, each

8 nonlinear constraint's projection is of the form 5 x 2, x 1 min where min is a constant depending on the text. In the second step, each constraint has the projection y 1, y 2 f h (x 2, x 1 ) where x 2 and x 1 are constant values after the rst step. Choosing an x-system solution during the rst step can't prevent the second step from nding a y- system solution as long as at least one solution for the whole system exists. Following this procedure, the constraint solving algorithms presented in [1] can be used together with nonlinear constraints without any modication of the algorithms. Optimal adaption of a text block's size to its text is thus possible without special methods provided for just this purpose. The only preconditions to make this procedure applicable are the ability to partition the original constraint system and the two properties mentioned above. Thus, this solving procedure isn't restricted to the NSD example. It can rather be used in the general case: the constraint system is partitioned into several subsystems consisting of merely linear constraints. The subsystems are linked by nonlinear constraints. Provided the nonlinear constraints have the two properties described above, solving the whole system can be performed by subsequently solving the subsystems while including appropriate projections of the nonlinear constraints. 5 Conclusions Interaction with diagrams causes specic problems that don't exist in widget based UIs. Approaches to solving these problems have been presented in this paper and are part of our diagram editor generator DEG. To cope with the diculties of dialog control when the user interacts with diagrams, event automata have been introduced. The model oers a limited form of nondeterminism being resolved by the execution engine. This is done in a way visible in the specication of an automaton and supports abstraction. The multiagent model surely is more general, but is not visual and doesn't oer help for the special diculties with diagrams. For a more general concept, event automata could be incorporated into the multiagent model such that an agent's state is represented by an event automaton. The structure of the event automata model ts better for this composition than Statecharts does. Since we have lots of objects in diagrams, it should be possible to compose them into semantically meaningful groups. The method for specication presented here is a bit more tedious than in the Synthesizer Generator, but also more general. Multiple selections are possible and the result is independent of the order of select actions. Furthermore, it has been shown how nonlinear constraints can be included into the layout algorithms for diagram objects described in [1]. The components described in this paper except from nonlinear constraints have been implemented for DEG, but some other parts are still missing. As soon as they have been added, interactive diagrams generated from a specication can be included into GUIs and ease the implementation of visual languages. References [1] M. Minas and G. Viehstaedt, \Specication of diagram editors providing layout adjustment with minimal change", in Proc IEEE Symposium on Visual Languages, pp. 324{329. IEEE Computer Society Press, [2] A. Morse and G. Reynolds, \Overcoming current growth limits in UI development", Communications of the ACM, vol. 36, pp. 73{81, Apr [3] T.W. Reps and T. Teitelbaum, The Synthesizer Generator, Springer, New York, [4] D. Harel, \Statecharts: A visual formalism for complex systems", Science of Computer Programming, vol. 8, pp. 231{274, [5] J.A. Larson, Interactive Software, Yourdon Press Computing Series. Yourdon Press, Englewood Clis, New Jersey, [6] L. Bass and J. Coutaz, Developing Software for the User Interface, The SEI Series in Software Engineering. Addison-Wesley, [7] R.J.K. Jacob, \A state transition diagram language for visual programming", Computer, vol. 18, pp. 51{59, Aug [8] A.I. Wasserman, \Extending state transition diagrams for the specication of human-computer interaction", IEEE Transactions on Software Engineering, vol. 11, pp. 699{713, Aug [9] G.E. Krasner and S.T. Pope, \A cookbook for using the model-view-controller user interface paradigm in Smalltalk-80", Journal of Object Oriented Programming, pp. 26{49, August/September [10] R.D. Hill, \Supporting concurrency, communication, and synchronization in human-computer interaction { The Sassafras UIMS", ACM Transactions on Graphics, vol. 5, pp. 179{210, In the following, we replace the width w by x 2, x1 and the height h by y1, y2 to emphasize the x- and y-attributes.