Event processing in Java: what happens when you click?

Transcription

1 Event processing in Java: what happens when you click? Alan Dix In the HCI book chapter 8 (fig 8.5, p. 298), notification-based user interface programming is described. Java uses this paradigm and you work by registering listeners which are called by the Java runtime system when users perform some action such as click on a button or select an item from a list. We are going to pick up this story after listeners have been attached to events and go through the series of things that happen. Underlying the way Java UIs work is the UI thread. This manages two complimentary paths of activity in the UI: (i) managing user interaction and dealing with user actions and (ii) updating the screen (painting). This model is common to many other GUI platforms including lower level Windows programming. Most of your code to do things will run in the UI thread as part of the response to user actions. However, if you use standard Swing components you may never explicitly write code that executes as part of the paint activity it is only when you need to produce a custom component and need to use direct graphics calls to draw lines, etc. that you may need to create a custom paint method. We ll go through the cycle of activities that typically occur when a user clicks the mouse. tracing the flow between these two paths of activity in the UI thread and your own code. Figure 1 shows an overview of this process and we ll go through each step in detail. Fig 1. the event-paint cycle

2 Stage 1 the user clicks When the user presses or releases a mouse button, moves the mouse or types on the keyboard an event is generated deep in the system. At the operating system level this is first channelled to the right application depending on what windows are visible, which application has control of the keyboard etc. Assuming this is your java application, this eventually ends up in the Java runtime environment, which does a similar job deciding which component the event should be directed to. It needs to take into account that components may be placed on top of one another (e.g. when a combo-box menu hides part of the panel beneath) or not be active (e.g. in tabbed panels). Stage 2 a listener is called Having found out which component is to receive the event, the Java runtime looks up the relevant registered Listener for the event. So, if you have added a MouseListener then this will be found if the event is a mouse press/release or if the mouse is dragged into or out of the component. If no listener is found for the event a default behaviour is performed sometimes to ignore it, sometimes to pass the event to the component containing the target (e.g. if the component has been added to a JPanel). If you have registered a listener object for the event, then the appropriate method is called. In the case of a mouse click for a MouseListener object, the mouseclicked() method is invoked and your code starts to execute. Stage 3 doing your own things Now your code gets to execute and this will typically mean updating some aspect of your internal state (or Model), setting variables, updating data structures etc. You may just be updating standard Swing components, perhaps setting putting a String into a JTextField however this effectively updates the state of these components. Note however, that your code inside the method is being run in the ui thread. This means that while it is executing no other user input can be processed (although events such as keypresses, mouse clicks etc. will be queued up to be dealt with later). This is quite a good thing if this were not the case and a second user action happened before the first was complete you would have the second event being processed while the first was half way through just imagine what would happen to your data structure Happily you are spared this problem, because there is a single ui thread all the events are serialised and the methods in your code to deal with them get executed one at a time in the right order. However, there is a counter problem: if you do lots of computation in your event handlers, the user interface will freeze until you are done (haven t you see applications just like that!). Normally this is not an issue if you are just updating the odd variable etc. However, if you do really large amounts of computation (e.g. run a simulation), or need to access external resources (read a file, access a database, grab a web resource), then there is a danger that the interface may hang. You can avoid a hung interface by having your own thread to perform complex calculations, wait for network things to happen, etc. but if you do this then you need to be careful about synchronising this with the UI thread but let s assume the actions to perform are simple!

3 Stage 4 calling repaint Normally the effect of the event is to change something that requires the screen to be updated. If not why not? If something has been done then the user needs to know about it! The possible exception would be where the event for some reason had no effect, perhaps clicking over an inactive button in which case does the button clearly show it is inactive? Assuming the screen does need to be updated, you may naturally feel you want your code to start writing to the screen: drawing lines, boxes, displaying text. However, in Java and many UI toolkits and environments you do not do this directly at this point. Instead, this is left to the paint activity. However, you do need to tell the runtime system that the screen requires updating and to do this you call the repaint() method on components that need to be redrawn. In the case where you are sub-classing a standard component (most likely JComponent or JPanel), this means you just run repaint() and the repaint method of this is called. Note that the repaint() method does not actually repaint the screen! In fact all it does is set an internal screen dirty flag that tells the UI thread that the screen needs to be updated. If you are using standard Swing components you may never call repaint() directly, but when, for example, you set the text in JTextField, internally the settext() method will call repaint(). Also if you use a Model-View-Controller model, you may again not call repaint() directly in your Listener, but it will update the Model, the Model will tell the View that it has changed and the View will call repaint()! Note that when you update several components, repaint() will be called several times. The system underneath keeps track of this and builds a list of all the parts of the screen that need to be repainted. Also, if you are calling repaint() and only a small part of your component has changed, you can give it a bounding rectangle to tell it that only a part of it needs to be repainted, that is specify a rectangle that includes all areas of the screen that need to be repainted. Stage 5 ui waiting Often repaint() is the last thing that happens in your listener, but need not be. However, when your listener has finished it returns. At this point the UI thread will catch up on any missed user events (if your listener did do lots of computation and took a long time!) calling the relevant listeners in order, but most often there are none and it simply waits for more user interaction. And if there are no user events to process Stage 6 the paint activity enters the action When the UI thread has no user events to process it is free to update the screen if necessary. It checks whether the dirty flag has been set and if it has, the screen needs updating. It needs to work out which portions of which components need to be repainted and then asks each component to draw itself on screen by calling its paint() method. If there are several overlapping components it will draw them backmost first, so that the foremost component gets drawn on top. Note that repainting may also occur when the events are internally generated, such as receiving a network message, or due to user actions that are not obviously to do

4 with the application, such as resizing a window, or exposing it by closing another window. Stage 7 component paint thyself Eventually your component gets to actually draw itself on screen. For standard Swing components this all happens in the Swing code, but if you want to do something special you can override the default paint method and write your own. In the case of a simple component you can override paint() directly, but if you are creating a custom component that may contain other components (e.g. if you want a standard button on you custom component), then instead you may override paintcomponent(). The default paint method calls this first to paint the background and then one by one calls the paint() method on its sub-components. Your paint method is passed a Graphics object. This is effectively a handle or way of accessing the portion of screen to paint to, although often is an off-screen buffer that is copied to the screen proper when you have finished. The Graphics object can be drawn onto with lines, geometric shapes, text and images (there be dragons!). The model while you are in paint() is of adding things one on top of another. If you draw some text and then draw a rectangle overlapping the text, the rectangle will cover the text (unless it is drawn in a translucent colour). However, note that if you draw a rectangle on screen when paint is called one time and do not draw it when it is called again, the original rectangle will disappear the model is that just before paint the relevant area of screen is wiped clean; you start with a blank canvas every time. This is why it important that you maintain a model of your internal state (whether this is a special class or just some variables), which you can refer to when painting the screen. In most toolkits including Java AWT/Swing, anything you draw is clipped to the region the paint thread wants redrawn. This means you do not have to worry about drawing things near the edge of the screen that might draw outside its borders, or when your window is partially obscured. However, when you have a very complex screen, you may want to use this fact and not bother to draw things that will fall outside the area being repainted. To do this you can look at the Graphics object and ask for its clipping region. However, you have to be careful to redraw everything that overlaps the region otherwise parts of things will disappear from screen. For even moderately complex screen layouts I usually find it easier to simply redraw everything. but do remember back-to-front drawing order. Stage 8 and so to rest The paint() method returns. If the Graphics object was actually pointing to a temporary off-screen buffer, this is copied to the screen and the paint activity waits for the screen to be again marked dirty by repaint(), and the UI thread waits all is peaceful in the world of the Java GUI until the next user interaction!

5 Unexpected events and the birthday surprise For simple single-user interfaces each user event gets processed and the screen updated before the next event arrives. People have a maximum rate at which they can type or click and the machine is fast! The exception is dragging and draggingrelated events such as slider movement, which can occur very rapidly, but even these are compressed a little by the runtime system. Instead of every single pixel movement it often clumps these together so you get a single movement event covering a substantial distance. However, if events can occur from outside (as in a chat program), or from things running independently inside (such as a thread doing lots of computation), you may get events occurring close to one another and not necessarily in the order you expect. The effects of this can sometimes be innocuous, but can also be disastrous. Imagine a network event occurs part way through your mouseclick handler. The network event is likely to be picked up by a different thread in your program that was listening to the network and may start some computation. Both are running concurrently and start to update the internal state at best things will go awry at worst your program may crash Similar things would happen if the network event happened part-way through your paint() method being executed. The paint would be unlikely to update the state, but may try to access state that is half-way through being modified by the network event handler again at best screen drawn with inconsistent data, at worst an Exception in the UI thread. When you test programs you often miss these unlikely events, but sooner or later they come back to haunt you! It may seem that given how little time the screen repainting, user event processing and other event handling takes, the likelihood of two things happening close enough to one another to interfere is very low. Unfortunately although the likelihood of any particular event hitting another is low, the likelihood that some event do so will is usually high. This is because of the birthday surprise property! As I demonstrated in the lecture this is a very impressive party trick. If there are around 30 people in a room you say I bet two of you have the same birthday. Each person thinks that with 30 people and 365 days in the year, it is pretty sparse so there are unlikely to be 2 people on the same day. In fact the opposite is true. Although for any particular person the likelihood of having the same birthday as any other person is low, and with 30 people only about a 1/12 chance that someone will have the same birthday as you, the birthdays have to be different for every possible pair of people and the number of pairs goes up with the square of the number of people. As we go through the people, person A says their birthday, then person B says theirs. There is only a 1/365 chance that this is the same as A. Then person C says their birthday and this time there is a 2/365 chance it is either the same as A or B. Person D has a 3/365 chance of having a clash. In order to have NO clashes each has to be different from ALL the rest. The exact probability of 30 people all having different birthdays (getting rid of the feb 29 birthdays before we start!) is: (1-1/365) * (1-2/365) * ( 1-3/365). * (1-29/365)

6 This is hard to calculate exactly (big numbers), but is approximately: exp( 30*29/2*365) = In general with N people exp( N*(N-1)/2*365) When N gets bigger than square root of 2*365 (that is about 27) this starts to get small with 30 people at least two times out of three you will win the bet Fig 2. birthday surprise number of pairs increases rapidly For events occurring the same surprise clashes occur. A short while ago I was reviewing some open-source ecommerce software. The software generated unique transaction ids to send to the credit card payment gateway. To do this it used the time in seconds (to be fair based on example code from the gateway provider!). The software was aimed at low volume sites, perhaps a few hundred transactions a day. they probably reasoned: 86 thousand seconds per day, say 500 transactions, no problem. But just like the birthdays the likelihood of clashes increase with the square of the number of transactions. In this case with more than 346 transactions a day there is a 50:50 chance of getting a clash any day this might mean a customer getting billed for the wrong thing, not charged at all, etc. I almost did the same once, but using milliseconds as id this time 86 million milliseconds a day. However, more then transactions and you are close to 50:50 for a hit! So, even if the number of events from different sources appears reasonably low, if you do not explicitly use some form of locking or semaphore to prevent simultaneous access to your internal state variables or model, sooner or later you will get problems. See also: Wolfram Research (2005). MathWorld Birthday Problem. (accessed 12/11/2005)