Games & Agents Lecture 2 Agents and Intelligence Jan Broersen
The subject of today I will give a very broad overview of different ideas from AI about endowing agents with intelligence. First: a movie about how the game-industry looks at intelligence
The AI in Killzone 2
Intelligence In the trailer: (Agent) intelligence: (1) enemies pin you down and rush up to you, (2) you do not see the same thing twice, (3) the environment is destructible (4) the AI responds to changes in the environment, (5) lean and peak system Multi-agent intelligence: buddies in the game that tell you to take cover, saving your skin In AI: Intelligence = intelligent behavior (we endorse Turing s test) Intelligent behavior = selecting the intelligent choices from all the ones that are feasible and performing them Agent theory: intelligence is a function from percepts to actions; the function can either be programmed or learned
Simple reflex agents
Simple Reflex Agent
Decision tree Conditionalization of actions is of the simplest kind In the tree: decisions at the squares depend only on observations at these squares. Cycle: sense, act
Are these used in computer games? Yes: these are the scripted agents Where do agent actions in scripts conditionalize on? Very simple information like the player being nearby However, intelligent choices should 1. be conditional on the state of the environment 2. be conditional on the state of other agents 3. be conditional on internal states 4. adapt after earlier experiences
Shortcomings simple reflex agents (1) No conditionalization on appropriate circumstances (2) No situation awareness (3) No looking ahead for decision making => agents need a model of the world, that is, we have to give them knowledge => agents need a goal in life, that is, we have to give them motivations (desires, intentions, obligations, etc.)
Knowledge-based agents
Reflex + State Agent
Decision tree Decisions conditionalize not directly on the percepts, but on the agent s model of the world (recalls what happened before and what it has done) How the world is, may now be derived instead of observed Allows for reasoning before acting Cycle: sense, reason, act
Knowledge-based agents Perceived as a major break-through in the late 50ies, just after the birth of artificial intelligence as a discipline (maybe the same brake-through still has to be realized in game-ai..) Model not only knowledge, but also lack thereof (uncertainty) Logic based approach 1: epistemic logic Logic based approach 2: non-monotonic reasoning Bayesian approach: beliefs as probabilities
Are these used in computer games? Hardly (do you know of examples?) However, if we agree the cognitive SOAR architecture gives us knowledge based agents, then we do have an interesting example. Paper: [It knows what you re going to do: adding anticipation to a Quakebot, John Laird]
A cognitive architecture: SOAR Close to ACT-R SOAR is claimed to be: a candidate unified theory of cognition (Newell, 1990), and an architecture for intelligent behaviour Decisions are never precompiled into uninterruptible sequences => continuous (and concurrent) interaction with the environment and with multiple internal processes.
The SOAR Architecture SOAR is an AI engine for making and executing decisions Basic objects of decision: operators (we would call them actions) Operators are either environmental actions, internal actions, complex actions, or abstract goals. SOAR is completely rule-based So, far, very much like 3APL/2APL
The SOAR Architecture Long-Term Memories Procedural/Skill Semantic/Concept Episodic Rule Learning Control Procedure Perception Semantic Learning Short-Term Memory Action Episodic Learning Body
But, unlike in 3APL/2APL: There is no deliberation or control language! Also there is no predefined order in which to apply rules: the system is completely data-driven. The current state ( working memory ) consists of perceptions, elaborations on it, and everything else derived by earlier rule applications. SOAR continually and concurrently proposes, selects and applies operator rules to the current state. Abstract (goal) operators that cannot be executed directly generate sub-states for which additional operators are proposed, selected and applied (remember HTN-planning)
Adding anticipation to a Quakebot Central idea: enable a Quakebot to project itself in its enemies to predict its behavior and to take advantage of that. Similar to a 1 ply deep minimax or alfa-beta algorithm (Chess, game theory) where the idea is also to use the same evaluation function for opponents. Add three new sets of rules to SOAR bots: 1. For deciding when to start predicting 2. For generating expected opponent behavior 3. For proposing when to stop predicting and for how to take advantage of the prediction (e.g. ambush)
When to start predicting? (task 1) Not always.. Decided by a special set of rules Which are up to the Quakebot designer..
Generating expected behavior (task 2) First the quake bot generates an expected internal state of its opponent, by applying special rules on his perceptions of the other (his position, health, armor level, current weapon). Then he applies his own behavioral rules on this state, thereby projecting himself in the other! The behavioral projection works on the generated internal state and projected environment, which is less computational intensive than the real thing.
Predictions, and how they are used (task 3) Predictions should not go on forever: special rules decide when to stop. For instance if an ambush is not possible due to environmental conditions or the timing. Predictions also stop when they encounter too much uncertainty (impasses in the prediction!). In the paper s set-up the technique is only used to hunt, ambush and deny power-ups.
Limitations Anticipations is restricted to situations deemed worthwhile by the programmer. So, tactics are not discovered but preprogrammed by the Quake bot designer. Alternative: always do predictions, and add general evaluation functions and planning too expensive, computationally.
Conclusions SOAR application to Quake SOAR quake bots use about 750 rules. Adding anticipation requires only 4 rules for deciding when to start predicting, 16 to generate an approximate opponent internal state, 5 to use the prediction. So adding anticipation turns out to be quite easy, from a programming point of view. But, computational much more intensive.
Shortcomings of knowledge-based agents Goals are implicit in the set of rules Agents decide without explicitly looking ahead in the tree
Goal-based Agent
Deliberation cycles for the different agent types Simple reflex agents: sense, act Reflex agents with state: sense, reason, act Goal-based agents: sense, update beliefs, select goals, select intentions, etc..., act
The tree view: planning
Are these agents used in computer games? Yes: path-planning using A* Knowledge: the network of waypoints Goal: finding a shortest route However, this is a very specific type of planning, because actions (movements) and goals (wanting to be somewhere else) themselves are very specific Knowledge about waypoints is simply assumed (and not learned through observation) Because the problem is so specific, we might even consider to calculate most of it offline for a given fixed environment..
Planning See course intelligent systems State space planning versus plan space planning (HTN planning) Planning under uncertainty (conformant planning, contingency planning, etc.) Main focus of research: controlling complexity, heuristics
BDI-theory Prominent theory: Belief-Desire-Intention Issue: how do all three depend on each other? Example: commitment strategies [Cohen and Levesque]: you keep an intention as long as you belief the associated desire to be attainable Example: realism: you do not desire things you believe to be true already; you do not believe things only because they match your desire (wishful thinking)
3APL/2APL Inspired by BDI Goal-base, Belief-base, and Plan-base (intentions) A language to specify the deliberation cycle Intuitions from Prolog
Shortcomings Knowledge is not fine grained Goals are not fine grained
Utility-based agents
Utility-based agent
Maximal expected utility Cycle: sense, find optimal tactic given beliefs, utilities, act Reasoning is now in terms of utilities, chances and look aheads: MEU, MDPs, PoMDPs
MDPs and PoMDPs Again see the theory explained in the course intelligent systems (Policies, Bellman equations, value iteration, policy iteration, etc.) I do not know of any example from computer games where this agent theory is applied (do you?)
Bayesian update Bayes' theorem adjusts probabilities given new evidence in the following way: Where H represents a specific hypothesis, which may or may not be some null hypothesis. P(H) is called the prior probability of H that was inferred before new evidence, E, became available. P(E H) is called the conditional probability of seeing the evidence E if the hypothesis H happens to be true. It is also called a likelihood function when it is considered as a function of H for fixed E. P(E) is called the marginal probability of E: the a priori probability of witnessing the new evidence E under all possible hypotheses. P(H E) is called the posterior probability of H given E.
The three prisoners Three prisoners A, B and C are told they have a 2/3 chance of being executed. At a certain point the decision is made by the director of the prison. Then the director visits prisoner A. A asks: will B or C be executed? The director answers: B What is A s chance of survival? Answer: unclear!
The three doors The Monty Hall dilemma Conditionalizing on extra information The chances for getting the car when not changing are 1/3. When changing, the chances are 2/3! (1/3 for initially correct and changing to the wrong door and 2/3 for initially wrong after which changing guarantees the good outcome).
Shortcomings However, No learning (like all of the previous) Or, is there? Any update of the model of the world can be seen as a learning action
Learning Agent Can be combined with any of the previous
Trends in Neural Networks There has always been the black box criticism towards neural networks (I call it the tweaking problem ) Current trend: add structure to neural networks How should such structure look like? Inspiration from biology? NEAT: the structure of the neural network evolves determined by a genetic algorithm Liquid state machines Neuro-symbolic computing
Rationality versus emotion
Some functions of emotions Frijda: emotions are (search) heuristics. Emotions may thus be said to guide planning. E.g. happiness may keep an agent on the right track, because it makes him selecting similar choices to reinforce his happiness. E.g. anger or distress may guide an agent into selecting choices he would otherwise not consider. Those choices maybe the solution to his problem. A thought: A* is an optimistic algorithm. Because of that it is complete (always finds a solution if it exists) and optimal (it will always find the best solution). Optimism seems to be related to positive emotions...
Emotions and rationality What is the relation between rationality and emotion? Both are involved in selecting (intelligent) decisions. For instance, rationally you estimate it is better to kill your enemy, or he will probably kill you. But, emotionally you feel extremely bad about killing. Or, rationally you estimate it is good to punish your child, emotionally you detest it. There is an example case of a man without emotion due to brain damage. He kept an average IQ, but could no longer decide on simple things like what to wear.
The Multi-Agent paradigm
Computing perspective Multi-agent systems constitute a new computational paradigm based on: Distribution of computational power Interconnection and communication Autonomy and Intelligence Delegation and distribution of control Anthropomorphic concepts
Software engineering perspective Multi-agent systems are a new software engineering paradigm. Analysis Design Implementation Test The aim is to provide high-level abstraction, to model and develop complex systems 70 s: Structural analysis methodology 80 s / 90 s: Object-oriented methodology 90 s / 00 s: Agent-oriented methodology
Artificial intelligence perspective Multi-agent systems and Artificial Intelligence do not coincide. Artificial Intelligence Planning Learning Vision Language understanding Multi-agent Systems Understand and model social intelligence and emergent behavior Interaction and Communication Social concepts: obligation, norms, responsibilities, etc. Optimal solutions can be obtained by co-ordination and co-operation Simulation can verify social and economic theory
Characteristics of multi-agent systems Multi-agent systems consist of a number of interacting autonomous agents. Multi-agent systems are designed to achieve some global goal. Agents need special abilities to cooperate, coordinate, and negotiate to achieve their objectives. Multi-agent systems are specified in terms of high-level abstract concepts such as role, permission, responsibility, and interaction. Applications for multi-agent systems are, for example, power management systems, transportation systems, and auctions.
Autonomous agents Autonomous agents are intelligent agents from a multi-agent perspective. Autonomous agents are active, social, and adaptable computer systems situated in some dynamic environment that autonomously perform actions to achieve their objectives. 1. Reactivity: responding to changes in the environment 2. Pro-activity (deliberativeness): goal-directed behavior 3. Social awareness: interaction and communication 4. Rationality: maximization of utilities Agents decide which action to perform based on their internal state. The internal state of agents can be specified in terms of high-level abstract concepts such as belief, desire, goal, intention, plan, and action.
The End