Artificial Intelligence in Virtual Reality. Introduction. Problems and possibilities

Transcription

1 Artificial Intelligence in Virtual Reality Daniel Sjölie VRlab, Umeå University Introduction Why do we want AI in VR? Simulation of humans (or animals) Interactive and realistic scenarios Intelligent Virtual Environments Problems and possibilities for AI in VR Simulated perception Dynamic behaviours Some examples Problems and possibilities Perception in Virtual Worlds Why simulate perception? The senses Active and passive sensing Detection and identification Perception solutions Decisions and control Plan or react? Emergence Emotions Behaviour solutions

2 Why simulate perception? Brute force alternative All data is already available in the virtual world. Realistic behaviour Characters should only react to events they would perceive in a real world. Efficiency You do not have to process events not perceived. Design modularity Separating perception from decision makes it possible to refine each part separately. The senses Sight The most important sense in most applications. Simple in principle, harder to make efficient. Many different solutions available. Sounds & smells These both propagate through a medium (air). Easy to create a simplified solution (radius). Hard to create a correct solution (medium). Touch & taste A matter of collision detection and triggers. Active and passive sensing Active sensing Active examination of the world. Generally affects the world. eg. sending out laser rays to judge distances. Passive sensing Just catch what comes your way and interpret it. eg. a camera catching the light. Virtual sensing No free possibility to have passive sensing. Need to simulate a medium to get this. Otherwise active examination of the world is necessary.

3 Detection and identification We need to detect and identify objects What detection and identification covers in general is not entirely clear but in VR there are distinct steps. Real detection Detection of objects should match the real world. Realism and performance is what matters. Virtual identification Virtual objects can include information about their identity. Basic identification can be essentialy automatic in VR. Perception solutions Identification is trivial once an object is detected Detection can be done in many ways Some approaches: Spatial subdivision (tiles) Ray-casting Image-rendering (with analysis) Simulated propagation Collision detection These approaches can often be combined Spatial subdivision Spatial subdivision is needed in some form Without it you must consider all objects. Tiles Probably the simplest form. Makes it possible to cache visible areas. Attractive for AI algorithms such as path-finding. Easy to add properties to the terrain. Could be integrated into graphics Spatial subdivision is often benefitial for the rendering of graphics. This could be adapted to enable reuse for perception.

4 Ray-casting Really two methods Cast rays to search for object Cast rays to test visibility of object Searching is generally not a good idea Must cast lots of rays to catch small objects Lots of rays are expensive Testing visibility works better Every ray is guaranteed to hit no wasted rays. Ray-casting really should be combined with spatial subdivision Image-rendering Not suited for VR without modifications! Rendering a view for each character and analyze the image throws away the identification advantage. Suited for testing of real world methods Vision limited to viewport Might be desirable in som cases (games) Modifications could be interesting Eg drawing with object ID s instead of colors. The z-buffer has interesting properties. Expensive! Rendering the scene once per character takes time! Simulated propagation Simulating the real world The goal is to make passive sensing possible. Propagation needs a medium Combine with spatial subdivision. The detection problem is largely isolated. Signal propagation is trivially parallelizable and could be executed in parallel with any AI. Can handle more than vision! Other approaches have problems here.

5 Collision detection Should rarely be necessary for perception Easy enough to implement if needed Plan or react Planning Without planning the avatars will lack the intelligence needed to simulate humans with goals and agendas. Reactions In large virtual worlds we want the avatars to be able to deal with events they cannot plan for. They need to react. How to combine these? Influence reactions with planning or force replanning when required Emergence Good stuff Makes complex behaviours manageable. Easy to experiment with. Well suited for simulations and research. Bad stuff Hard to control. Overkill for simple behaviours. Not well suited for scenarios.

6 Emotions Simulation of human behaviour requires simulation of emotions Emotions also work well in many cases for nonhuman behaviours Balance is needed Emotions should influence the behaviour but rarely control it Behaviour solutions Two basic steps Prioritize behaviours Combine behaviours Behaviour prioritizing Subsumption. Arrange behaviours into a hiearchy where lower behaviours can be overridden. Weights. Give each behaviour a weight without removing.

7 Behaviour combining Winner takes all. The behaviour with the most weight decides all. Voting. All behaviours vote, with differing influense, on each action that could be taken. Some examples ViCrowd, Lig PeopleShop, Boston Dynamics Mission Rehearsal Exercise Games ReplicantMind ViCrowd Created at Lig in Switzerland Focus on large crowds by working with groups and crowds as units. Uses hierarchies of autonomy to control the complexity of individuals.

8 ViCrowd Uses plan and re-plan. Each individual has a calculated path to follow within the group. The primary unit is the group. Individuals can change group or take command of a group. PeopleShop Created by Boston Dynamics Focus on control to make it easy to create specific scenarios More guided animation than simulation MRE Mission Rehearsal Exercise (in Bosnia) Created for the U.S. Army Simulates emotions to create engaging and realistic scenarios May be the most complete solution today

9 Games AI has become a selling feature The Sims is one prime example of successful game AI. The simshave a number of desires and needs that makes up the basis for their decision making. The objects contain information on their effect. Games In Black & White the creatures creates models of the minds of players and other creatures. Simulated perception is important in games like Deus Ex and Splinter cell. ReplicantMind An AI toolkit created at VRlab. One base class, Entity, for all intelligent objects. Each entity can have any number of desires. Desires are managed by processes. Processes work by reacting to perceived entities.

10 ReplicantMind - voting The actions to execute are selected by tasks. Tasks are created by processes and motivated by desires. Tasks suggest actions that suit them. Tasks vote on suggested actions and the action with most votes wins. The vote of each task is weighed according to the importance of the desire motivating the task. ReplicantMind example An entity called man has a desire to eat and an eat process. This eat process reacts to a spotted food entity by creating an approach task and motivating it with the desire to eat. The approach task suggests an action moving towards the spotted food and votes for any action taking us towards the food. This all blends with any other active processes, desires and tasks. ReplicantMind - tasks Much of the complexity ends up in the tasks. The voting can be simplified by having the effect of the action predicted and let the task evaluate the resulting situation. Getting the tasks to reach satisfying compromises can be challenging. One relatively easy way for a task to compromise is to check the last selected action and adapt.

11 The End EOF