C0600 Technical Report. Computer Animation in Java Emergent Behaviour in Autonomous Agents

Transcription

1 C0600 Technical Report Computer Animation in Java Emergent Behaviour in Autonomous Agents James Cogley Paul Hempseed Will Smith

2 Computer Animation in Java Emergent Behaviour in Autonomous Agents James Cogley University of Kent Computing Laboratory Canterbury, CT2 7HR Paul Hempseed University of Kent Computing Laboratory Canterbury, CT2 7HR Will Smith University of Kent Computing Laboratory Canterbury, CT2 7HR Abstract This details the research and implementation of an application to demonstrate the use of emergent behaviour (i.e. flocking) in autonomous agents. It shows that with additions to the basic flocking rules, a complex 3D visualisation of a scenario can be created. 1. Introduction Emergent behaviour refers to a group of simple entities working together to form more complex behaviours. The main sections of the work were the simulation (how the boids behave, how they work together, how they react to challenges), the visualisation (taking the simulation and environment and displaying them, the physics etc.) and the environment (designing of scenarios, obstacles) The main areas approached to complete this such as the simulation; the visualization and the environment are described in more detail in the report below. 2. Background In 1986 Craig Reynolds demonstrated a computer model for simulating the co-ordinated movement of animals such as schools of fish or flocks of birds which he consequently dubbed Boids (1). Since then there have been many examples of Boids and flocking being applied in the areas of behavioural animation, artificial life and individual based models. Behavioural animation is a procedural animation technique where an autonomous character determines its own action within certain constraints that are imposed by the animator. Since its feature film debut in 1992 (2) this technique has rapidly gained popularity in the film industry especially in recent years with more and more films using computer generated herds of stampeding wildebeests (3), hordes of alien insects (4) and even entire computer generated armies (5). Artificial life is a field of study in which attempts to model living systems, processes and their evolution through the use of computer simulations, robotics and biochemistry (6). Individual based models (IBMs) are simulations that are based on global consequences of local interactions (7). Individual based models can also be described as entity or agent based models; where the individual/entity/agent might represent a pedestrian in a crowd, or an animal in a particular environment or a vehicle in traffic. Because an individual based model is a collection of interacting parts it can also considered to be a subset of multi-agent-systems however IBMs can be distinguished by the fact that each agent represents an autonomous in the simulation. 3. Aims The aim was to create a demonstration of a flocking algorithm illustrated in a computer animation while allowing some degree of interactivity with the environment. The visualisation would be used to demonstrate complex crowd behaviours in an environment involving obstacles, attractors and repulsers. The agents used would follow the set rules of flocking and when these simple behavioural rules are used by multiple agents they should combine to create a complex emergent behaviour. The environment in which these agents exist had to provide enough stimuli to influence the boid behaviour but also present them with enough freedom to fully utilise the flocking rules. Initially the environment would be visualised in 2D which would eventually be developed into 3D. For further reading on the groups aims please refer to the Project Abstract (8) and the Requirements (9), (10) documentation.

3 4. Design and Implementation The design of the application went through a number of significant changes between iteration one and two as the focus of the project shifted and presented a number of challenges in both the design and implementation. 4.1 System Overview One of the very early design decisions was to treat the simulation and the visualisation as two distinctly separate parts and try to keep the simulation, as far as is possible, independent of the final visualisation. This gave the freedom to develop the visualisation using whatever method was best suited to the task and, more importantly, if things were not working out to change the method used while minimising any impact on the development of the simulation. This approach led to the application taking on a layered structure which is discussed further in section [4.1.2 Layered Structure] System Structure. The simulation package contains classes responsible for modelling the environment and everything within it; this includes Obstacles, Emitters, Attractors, Crowds and Boids. It also runs the flocking algorithms and calculates the forces that need to be applied to the Boids; the procedure for doing this changed significantly between Iteration One and Two. In iteration one the simulation class acted as a control class and was responsible for updating everything Simulation within the simulation, the visualisation would create an instance of a simulation and call updates on it as needed. This approach, while working well in iteration one, proved to be too inflexible for the features planned in iteration two. The simulation class was removed and the visualisation took over as the control class, using the classes in the simulation package as building blocks to create multiple complex scenarios The Layered System. Figure 1 shows a simplified overview of the application; it shows how the simulation encapsulates all of the entities that are present in a scenario. Entities that are rendered in the scene are extended in the visualisation package to include information about the visual representation of the object and how it will appear in the scene, as well as location information, information about translations and rotations and where it is located in the scene graph. Initially the Obstacle, Emitter, Attractor and Boid classes were intended to have their own r (for rendered) class in the visualisation package (e.g. rboid, robstacle etc.) which would contain information on how the object would be rendered in the visualisation. However as the Boid class was the only one intended to have a more complex appearance it was often easier to add the required fields to the original simulation class than to create a r class. While being easier this unfortunately had the effect of binding the simulation classes more tightly to the visualisation as much of the information was specific to the visualisation and the 3D API used. Visualisation Brain Boid rboid Visualisation Behaviours Crowd Scenario Environment GUI Neighbourhood Attractor Obstacles rattractor robstacle Emitters remitter Figure 1 - High Level View of the System

4 5. The Visualisation The visualisation is more than the front-end to the simulation; the visualisation takes the tools/building blocks from the simulation and uses them to construct complex scenarios demonstrating the flexibility of the simulation. Not only that it allows the user to interact with the scenarios, modifying them and observing the results. 5.1 Java 3D vs. jmonkeyengine From the very beginning there was a desire for the visualisation to be created in 3D (if only for the aesthetics); however it was not clear how feasible a 3D environment would be as the group lacked knowledge and experience of developing such an application. For the Visualisation a solution was needed that would enable the rapid creation of prototypes that could be easily expanded into full 3D if required (at a later date), that wasn t too difficult to learn and performed well Java 3D. Java 3D is a 3D API created by Sun to enable the creation of three dimensional graphics applications and internet-based 3D applets (11). Java 3D has a number of benefits; it has a large number of tutorials and documentation available on the internet. It has a number of official and community based geometry loaders available. And the general purpose nature of the API means it can be used to create a wide variety of applications. However because Java 3D tries to be everything to everyone the API seems to be unwieldy and the workflow appears to be restrictive jmonkeyengine. The jmonkeyengine (jme) is a high performance scene graph based API aimed at both hobbyist and commercial games developers (12). jme offers a number of attractive features, among them are; a wide variety of geometry loaders, allow animated meshes to be loaded and supporting both skeletal and vertex based animation. Optional packages such as the jme-physics package that provides Java wrappers for a number of physics engines, including ODE (13). The more focussed nature of the API meant that while it wasn t any easier to learn than Java3D there was less to learn API Conclusion. With the criteria mentioned in [5.1] in mind jmonkeyengine was chosen to develop the visualisation on; its more focussed approach and lighter-weight API combined with the optional packages and the ability to rapidly prototype ideas made it the best choice Visualising the Simulation As has been previously mentioned it is the role of the visualisation to provide a visual representation of the simulation; additionally the requirements state that it should provide an interface for the user to interact with the simulation Multi-Threading vs. Single Threaded. The jmonkeyengine is built upon the LWJGL (Light Weight Java Game Library) (12) which uses OpenGL (14) as a renderer. OpenGL is primarily designed for use in a single thread; as such the frameworks (such as SimpleGame etc.) within jme are also designed to work in a single thread. However in an attempt to regain some of the performance lost through the use of the physics engine, research into a multithreaded approach was carried out. The thinking being that if the Logic, Rendering and Physics could be split into separate threads it would remove, or at least, alleviate the drain on processor time caused by the physics engine. Unfortunately the physics engine was not designed for multi-threading either, and it proved impossible to keep the scene graph and the physics graph synchronised without modifying the physics engine, so the idea of multithreading had to be abandoned. 5.3 Visualising the Environment The graphical side of the environment interacts with the Visualisation class; this is represented on screen as simple blocks or cuboids, of various sizes in different locations. The first Iteration consisted of just four walls which had two purposes, to keep the boids in a basic environment in order for experiments to be run on their behaviour but also to make it easier to find any problems when creating and positioning obstacles. In the second iteration a train station was used to add more depth to the simulation. As obstacles could only be created as blocks it meant either the boids had to climb them like steps or some other obstacle shape be used instead. By using MilkShape (15) the two-stage bridge could be created (see Figure 2) that would allow boids to walk over it, reaching the other platform without trying to avoid it like a normal obstacle. From this, other obstacles were generated such as trains which can be added and placed in the same way as other obstacles but are visualised as complex 3D geometry rather than simple blocks.

5 Figure 2 - The Bridge Obstacle in Milkshape 3D 6. GUI The specifications for the visualisation stated that it should also provide an interface for the user to interact and modify the simulation. In iteration one the main emphasis was to develop the flocking algorithms and develop the simulation. In iteration two the focus shifted to developing the visualisation which presented several choices when it came to implementing a GUI. 6.1 SWING vs. jmedesktop vs. The Rest Designing and implementing the GUI component for the application presented some interesting decisions. As the initial designs for the interface were along the lines of a CAD program the obvious place to start was embedding the jme renderer into a standard Java swing application. That in itself proved to be a straightforward task, however things were not that simple; performance took a big hit (over 100fps slower than before) and on top of that the physics engine had issues running within the swing thread (by this time we were committed to using the physics engine) which ruled out this method. The next method of implementing a GUI was using the FengGUI packages; FengGUI is a graphical user interface (GUI) application programming interface (API) based on OpenGL. (16). In its favour the FengGUI interface had little or no effect on performance and seemed fairly easy to implement; however it did mean learning a new API and having to create an input handler to capture mouse and keyboard and the default theme was not particularly appealing. The final method investigated (17) was using the jmedesktop. The jmedesktop allows the rendering of Swing components onto a special node in the scene. The jmedesktop cost a few frames per second but not enough to rule it out. The Swing GUI runs in its own thread and can communicate with jme through the jme action handlers and events. 6.2 GUI Development and the Editor The requirement for interactivity with the simulation always intended for the user to be able to interrogate and modify properties of objects in the environment. In iteration one this requirement was delayed until iteration two while the concentration was on simulation. Iteration two gave the chance to explore a number of ways of enabling the user to modify the simulation on-the-fly. The step towards this goal was the addition of the Flocking panel; this panel displayed the behaviours, their strengths and their effective ranges (if applicable). This enabled the user to change the

6 balance between the forces generated by the flocking behaviours and observe the effects in the simulation. The next step was to add a rudimentary mapeditor (see Figure 3) that gave the user the ability to create simple scenarios for themselves by placing a series of pre-defined building blocks; Emitters (that create boids), Crowds (that contain boids, and tell them what attractor they are attracted to), Attractors (that give boids somewhere to go) and Obstacles (that give the boid something to avoid). Figure 3 - The Iteration Two Editor 7. Flocking and Behaviours The decision process for the simulation was a trade-off between the emergent behaviours of flocking, and the pre-defined certainties of pathfinding. It was decided that path-finding had been proven to work in the past and would not be a suitable system to test behaviours, as people do not move in straight lines, there is a degree of randomness that cannot be easily built into a pathfinding approach. Flocking on the other hand is a common way of showing emergent behaviours. It was chosen as the initial primary factor in controlling the boids in order to give them a natural decision process, without the constraints that path-finding has, so that hopefully they would learn and work as a group of boids to complete tasks, without specific knowledge of what those tasks would be. In Iteration 1, a purely flocking system was used to control the boids (20). They would use the behaviours of separation, alignment, and cohesion, as well as an extra rule to be attracted to attractors. This simplistic system had problems. It was found that certain behaviours were cancelling each other out. (21) For example, the cohesion and separation would often have the inverse velocity change of each other, making the effects of them nullified, and making alignment the overriding force on the boids, making them move slowly in formation. There were also problems that the behaviours all had the same weighting throughout the running of the simulation, so even when certain behaviours were required to be made more important (such as when boids were very close to an obstacle and heading towards it) they could do little to slow themselves down. For Iteration 2 therefore, it was decided to make the boids behaviours more complex. Each boid had a brain which would call a number of behaviours. (22) These behaviours had different weightings, which can be changed in runtime. This can be done in two ways. One is by the user using the GUI (Flocking Tab). It is also changed by the program when certain critical situations occur. For example, if a boid is heading toward a collision imminently, the weighting of that behaviour is increased to help it deal with the

7 problem and increase the probability of the collision being avoided. To help boids getting out of loops of behaviour, or getting stuck, random offsets were added to the behaviours, so that there would always be small differences in the decisions the boids would make, keeping them moving and helping boids avoid getting stuck. These additions and alterations to the flocking system gave the boids more independence in the sense of small movements, while at the same time making sure they followed the general rule of flocking. So instead of looking like a flock of birds moving in tandem, they looked more like a group of people walking through a train station. In terms of suitability, flocking alone was not a suitable approach. But, when made more complex with seek behaviours amongst others, it was possible to use flocking to create a realistic simulation. With more time, the aim was to have developed an interchangeable brain system with many more behaviours, which could call upon any mixture of these behaviours depending on the boids current state. With this system, due to the greater possibilities for action the boids can take, in terms of the different combinations of behaviours they can use, the possibility to make the boids more realistic is increased. The influences of these behaviours would continue to be independent and changeable in runtime, as would the list of behaviours the boid was using in different states of existence (near wall, near attractor, no boids nearby etc). 8. Collision Detection Management Collision detection management deals with the obstacle layout and positioning within the environment. In order for boids to make realistic alterations to their direction and velocity they must know whether or not they are about to collide with an obstacle. This was achieved by storing the locations of all obstacles in an Array List which acts as a grid representation of the environment, whenever a boid is about to move it checks the relevant grid space in the Array List to see if it contains an obstacle or not. What makes this important is that the boids have no knowledge of their surroundings; they simply react to what is in their path. The benefit of using an Array List is that it can be quickly searched by the boid using its velocity to predict its next vector. This method could have been changed to a tree structure to make the search times more efficient but due to time constraints and the fact that the current method still worked without fault meant that it was left unaltered. The final iteration uses a much more complex scenario which acted as a useful test of the collision detection system; this is made up of two platforms and a two-stage bridge connecting them, as well as a ticket office and gates. The main problem exists with the bridge as boids would simply identify it as an obstacle rather than a possible path. To overcome this, the bridge was not added to the list of obstacles and the physics engine was left to deal with the boid interaction. By ignoring the bridge the boids move over it as if they were moving on the same level. Other obstacles such as the train were not added to the list of obstacles because of the need to update the list every time they move. It was much more sensible to get the boids to avoid the railway lines instead just as a human would in real life. 9. User Testing and Performance The project fair presented the opportunity to gain some feedback from people outside of the project group. Visitors were keen to view the demonstration and all of the participants were impressed with the visual side of the project. However they were a little more hesitant when it came to using the application, finding the layout of interface to be difficult to understand and the process of creating a scenario cumbersome. It also became apparent during this session that the application requires a method for resetting the current scenario; as currently the application must be shut down and re-started in order to create a new scenario. The performance analysis (23) indicates that generally the application performs well on reasonably modern hardware. Even with the more complex, animated boids the rendering of the scene is still rendered at an acceptable speed. However the analysis does highlight some performance issues with the obstacle avoidance methods. 10. Conclusions From this report, it is felt that the following conclusions can be made about the different sections of the work. With the Visualisation, the visual appeal was impressive, and all the main requirements were achieved. However, some requirements were not up to the standards set at the beginning of the iteration. The editor was unable to create complex scenarios in a reasonable length of time, and was clumsy to use. With work it could be made to create bigger sections of environment quicker, but as it currently stands it is impractical. On the Environment side, it is felt that the methods it uses are very efficient; making it suitable to handle an even larger scenario than the train station it is currently used for. However, there are some reliability issues, particularly with the addobstacle method. It can come into difficulties in certain one-off situations due to an algorithmic inaccuracy.

8 With the Simulation, it was proven that flocking mixed with other behaviours can create a realistic simulation of a complex scenario. However, further optimisation of the methods in the simulation is required if a very large scenario was to be created. This could be done by the addition of a more efficient special partitioning system. 11. Further Development It was not possible to implement all of the planned features in the time allotted. Given more time it would have possible to implement some of the higher priority features, as discussed in the following sections Environment Loader / XML In the development of the train station scenario, it was recognised that the scenarios being created became more complex. It was also recognised that for the end-user of the system, the functionality of the program would be quite limited. It was decided that creating an XML Schema for the purpose of creating complex environments, which could then be imported into the program would be the best approach to tackling this problem. Unfortunately, due to time constraints the system was not completed. However, the XML Schema and an example XML environment file were made (18), and shows that with more time it would have been an attractive option to further the capabilities of the program Environment Builder Unfortunately due to time constraints many of the features planned for the editor were not implemented, or were not completed. Given time for a third iteration there were plans for providing the user with more pre-defined obstacle shapes (such as ramps and bridges), the ability to stack obstacles to increase their height and to place them on differing levels (to create stairs for example) and the option to import obstacles from external files. Further debugging information was also planned but not implemented in time, for example; lines to show a boids current destination, boid trails to show where the boids have been highlighting heavy traffic areas and attaching the camera to a boid enabling the user to easily track a single boids path through the scenario. 12. Acknowledgements We would like to thank our project supervisor, Dr Peter Kenny at the University of Kent, for his guidance and support throughout the course of the project. We would also like to acknowledge the members of the jmonkeyengine community for their work on the jmonkeyengine and finally Craig Reynolds for his work on Flocking (1) and Steering Behaviours (19). 13. References 1. Flocks, Herds, and Schools: A Distributed Behavioral Model, in Computer Graphics. Reynolds, C. W. s.l. : SIGGRAPH '87, pp Burton, Tim. Batman Returns. Warner Bros. Pictures, Roger Allers, Rob Minkoff. The Lion King. Disney, Verhoeven, Paul. Starship Troopers. Tristar Pictures, Jackson, Peter. The Lord of the Rings: The Two Towers. New Line Cinema, Unknown. dictonary.com. dictonary.com. [Online] [Cited: 25 March 2008.] life. 7. Reynolds, Craig. Individual-Based Models. [Online] 28 September [Cited: 25 March 2008.] 8. Cogley, James. Project Abstract Smith, Will. Iteration One Specification Iteration Two Specification Sun Microsystems, Inc. Java SE Desktop Technologies - Java 3D API. Java Technology. [Online] Sun Microsystems, Inc., [Cited: 25 March 2008.] 3d/. 12. Powell, Mark. About jmonkeyengine. jmonkeyengine.com. [Online] 24 January [Cited: 25 March 2008.] ontent&task=view&id=21&itemid= Smith, Russell. Open Dynamics Engine. Open Dynamics Engine. [Online] 28 May [Cited: 25 March 2008.] lwjgl.org. lwjgl.org. lwjgl.org. [Online] [Cited: 25 March 2008.] Ciragan, Mete. Chumbalum Soft. Chumbalum Soft. [Online] Chumbalum Soft, [Cited: 25 March 2008.] Schaback, Jjohannes. FengGUI. FengGUI. [Online] [Cited: 25 March 2008.] Cogley, James. GUI Design Hempseed, Paul. The Scenario System Steering Behaviors For Autonomous Characters. Reynolds, C. W. San Jose, California : Miller

9 Freeman Game Group, Game Developers Conference. pp Cogley, James. Flocking Algorithm Hempseed, Paul. Iteration One Simulation Evaluation Cogley, James. Behaviour System Figure Hempseed, Paul. Performance Analysis 2008.