An approach towards building human behavior models automatically by observation

Transcription

1 An approach towards building human behavior models automatically by observation Hans K. Fernlund and Avelino J. Gonzalez Intelligent Systems Laboratory School of Electrical Engineering and Computer Science University of Central Florida Orlando, FL Abstract Modeling human behavior could be complicated and expensive. To be able to reduce these costs, new methodologies and tools must be developed that automate the creation of building human behavior models. We describe in this paper one way to accomplish this through the use of Genetic Programming in conjunction with Context-Based Reasoning (CxBR). Context-Based Reasoning is based on the concept that humans think and act in terms of contexts. This approach has been proven to work well in simulated environments. Genetic Programming (GP) addresses computer programs that evolve new, better programs by themselves, i.e. automatic programming. GP will genetically breed a population of computer programs using operations and selection mechanisms inspired by Darwin's evolution theory. The result of the genetic process is a computer program that will solve most types of predefined problems in almost any area such as classification, planning, mathematics, optimizing or control. This paper presents a new approach for automatically creating human behavior models. To automate the creation process, learning from observation is used. This strategy learns the behavior of a subject matter expert by merely observing his behavior. The need for a methodology like this has already been identified for some applications, e.g. after-action-review programs, modeling human behavior in battlefield simulations and in street traffic flow simulators. Introduction Building human behavior models is very complex and time-consuming. A very intricate task is to extract and process the information from the subject matter expert. To get the subject to express the behavior in an acceptable manner, and then analyze the information and later implement it in the model are potential sources of misinterpretation. It is almost impossible to develop a mathematical formalism of human behavior, and the cost and effort to build good models can be very high. In the real world, there often exist problem domains where the knowledge might be incomplete, imprecise or even conflicting. Often, the models are built on inflexible doctrines. This can cause the entities to behave too perfectly without human similarities (Henninger et al., 2000). It has also been shown that the manual routines taught by the experts, are not necessarily the routines used by the experts themselves (Deutsch, 1993). The use of a learning system that could automatically extract knowledge and construct a behavior model could reduce the problems mentioned above. If the system would be able to monitor a human s behavior and automatically build a behavior model the development complexity would dramatically be relaxed. This paper presents an approach to building human behavior models automatically. The approach merges Context-Based Reasoning (CxBR) and Genetic Programming (GP) to arrive a methodology called Learning by Observation, which will learn the behavior of a human merely by observation. First the objective of Learning by Observation is addressed and the theories of

2 CxBR and GP are examined and finally the conclusions and advantages of merging those methodologies are presented. Learning by Observation The term Learning by Observation has its roots in biology. Studies have shown that humans fully develop observational learning by the age of 24 months (Abravanel, Ferguson, 1998). By that age, children can easily learn a simple task by observing another person performing the task. Inspired by how humans and other mammals seem to learn by observation, the machine learning community has developed a large number of theories on learning applied to different areas. The interest in this research is to investigate learning human behavior by observation. The intent is not only to use the observations to learn, but rather to learn the behavior of the observed entity. Thus, interest herein is to observe a human in action to be able to model his behavior. A number of advantages could be gained by using learning by observation instead of traditional knowledge acquisition and development methods. Reduce time and cost of development, debugging and maintenance. More accurate, realistic and refined representation of human behavior. Potential to incorporate learning from experience. Potential to incorporate new features of humanness, such as emotions. Relaxes the need for programming skills in the part of the operators. Reduction of problem domains. Develop simulated entities in real time. Several researchers propose learning by observation as a mean to overcome the knowledge acquisition bottleneck (Gonzalez et al., 1998; van Lent and Laird, 1998; Schaal, 1999). Most of the AI literature refers to learning by observation as a method of learning the behavior of another entity by observing its action. Note that learning by observation is a direction on how the data for learning is to be collected - through observation. It does not make any statements on what learning paradigms to use or what type of learning takes place. Since this research focuses on automatically modeling agents with human behavior, our definition of learning by observation is as follows: The agent shall adopt the behavior of the observed entity only through the use of data collected by means of observation. Models of human behavior encompass a lot of different humanness features. These features range from rather basic behavior as goal-driven behavior, planning, human reaction times and limitations, to more complex behavior as overlapping of performance of multiple high-level tasks, multi-agent coordination, communication, temporal reasoning and maintaining episodic memory. To create a model with these features there must exist an efficient modeling framework. Context-Based Reasoning (CxBR) was proposed by Gonzalez and Ahlers (1998) to have many of these features. The Context-Based representation in CxBR is modeled from the pattern of human behavior. Using CxBR as a framework satisfies the prerequisites for implementation of intelligent behavior. If the model is to be created automatically the framework also needs to be equipped with a learning paradigm that will work in conjunction with the framework without disturbing the supported human features. The proposed learning paradigm is Genetic Programming (GP) and this paper will show that CxBR and GP have the right prerequisites for developing a tool that can automatically build human behavior models. Context-Based Reasoning Gonzalez and Ahlers (1994) presented Context-Based Reasoning (CxBR) as a technique that can efficiently model the behavior of humans in intelligent agents. Further results showed that it is especially well suited to modeling tactical behavior. CxBR is based on the idea that: A recognized situation calls for a set of actions and procedures that properly address the current situation. As a mission evolves, a transition to another set of actions and procedures may be required to address the new situation. Things that are likely to happen while under the current situation are limited by the current situation itself. CxBR encapsulates knowledge about appropriate actions and/or procedures as well as compatible new situations into hierarchically-organized contexts. A sample hierarchy is depicted in figure 1. Mission Contexts define the mission to be undertaken by the agent. While it does not control the agent per se, the Mission Context defines the scope of the mission, its goals, the plan, and the constraints imposed (time, weather, rules of engagement, etc). The Main Context is the primary control element for the agent. It contains functions, rules and a list of compatible next Main Contexts. Identification of a new situation can now be simplified because only a limited number of all situations are possible under the currently active context. Sub-Contexts are abstractions of functions performed by the Main Context which may be too complex for one function, or that may be employed by

3 other Main Contexts. This encourages re-usability. Sub- Contexts are activated by rules in the active Main Context. They will de-activate themselves upon completion of their actions. MAIN CONTEXT1 Sub-Context1 Mission Context Figure 1 Context Hierarchy Main Context2 Sub-Context2 One and only one specific Main Context is always active for each agent, making it the sole controller of the agent. When the situation changes, a transition to another Main Context may be required to properly address the emerging situation. For example, the automobile may enter an interstate highway, requiring a transition to an InterstateDriving Main Context. Transitions between contexts are triggered by events in the environment some planned, others unplanned. Expert performers are able to recognize and identify the transition points quickly and effectively. CxBR is a very intuitive, efficient and effective representation technique for human behavior. For one, CxBR was specifically designed to model tactical human behavior. As such, it provides the important hierarchical organization of contexts. A full description of CxBR can be found in Gonzalez and Ahlers (1998). This concept has received significant interest from other researchers in the technical literature. Turner (1993, 1999), Brezillion (1999) and Bass (1996) have all independently developed context-based approaches to modeling human behavior. CxBR Theory The knowledge constituting the agent s intelligent behavior in CxBR is stored in the different contexts, i.e. Mission Contexts, Main Contexts and Sub-Contexts. The collection of contexts is called the context base. If the CxBR model is going to be useful, the context base needs to be equipped with at least an inference engine and a fact base. The format of the knowledge is not fixed in CxBR; only the knowledge structure is defined in CxBR. When building an intelligent agent with CxBR, the most common knowledge representation paradigm consists of if-then rules and functions. These functions could be predefined in the programming language package or user-defined command sequences. A special form of a command sequence is a context at a lower level, e.g., Sub-Sub- Context. A Sub-Context could itself also have a Sub- Context and henceforth the term Sub-context refers to a context at the next lower CxBR s hierarchical level. If we do not distinguish between functions and operators, we could define a function set, F={f 1, f 2,, f m }, that processes the information to deliver intelligent actions. The function set could exist of the following types (we use C/C++ notation for our examples): Arithmetic operations: +,-,*, Relations: <,>,==, Mathematical functions: cos, sin, pow, Boolean operators: and, or, Command sequences: distance_to(), Conditionals: if, else, Iterations: do, while, for, SubContexts, TrafficLightContext(), The function set is only useful if it can process information. Information has value and is stored in variables or constants. These information storages are called terminals and the information available is then defined in a terminal set, T={t 1, t 2,, t n }. Note that some of the values in the terminal set could originate from sensor reading and are therefore time or instance dependent. The Mission Context, MC, mainly defines goals, G, and plans, P. Plans are realizations to reach the goals for one or many intelligent agents. The action specified in the Mission Context is mainly to let the plans invoke Main Contexts in an appropriate sequence so the goals could be fulfilled. A set of sentinel rules, R, determine which Main Context shall be active. At least one default Main Context must be specified as a part of the sentinel rules in the Mission Context. A specific mission is now defined as: MC y ={G, P, R} The contexts from the Main Context level down in the hierarchy have all the same structure. The knowledge contained in a Context could be described as a set of actions, A, and a set of sentinel rules, R. The two main characteristics of a context are that it knows what action to take and it knows which context is most suitable for the current situation, i.e., situational awareness. The actions set describes how the entity will behave within the present situation, i.e. active context. The sentinel rules determine the active context. The active context could remain active or control could be shifted to another context. Now a specific context, C x, could be described as:

4 C x ={A, R}; A={F, T}, R={F, T} Both the action set, A, and the set of sentinel rules, R, contain a function set, F, and a terminal set, T. The only restriction of the function set of the sentinel rules is that the type Sub-context is not allowed. The F marks this slight restriction in the function set of the sentinel rules. Genetic Programming Genetic Programming (GP) is developed from Genetic Algorithms and both are stochastic search algorithms. The search process looks for the most suitable program that will solve the problem at hand. The target system for the GP could be a CPU, compiler, simulation or anything else that could execute the pre-defined instructions, from now on referred to as a program. GP evolves source code representing a program that can address a specific problem. This makes it very suitable for use together with CxBR. GP can build complete software programs that support even further development of the learning process to construct the internal structure of the CxBR, i.e., the context-base. To make GP work, some basic requirements must be satisfied. First, we need to have a set of individuals, i.e., programs that represent different solutions to the problem. Furthermore, all the individuals need to be evaluated in some manner as to what degree they are able to solve the problem. The features of the individuals with better suitability would preferably be preserved and survive or breed new individuals to the next generation. The next GPstep would be to evolve the individuals, i.e., reproduction, in some manner to preserve the good features and develop even better individuals. The genetic operators, e.g., crossover and mutation, will support the development and evolution of the individuals. Evolving a program with GP can be described in five steps: Create an initial population of programs (usually randomly generated). Evaluate the performance of each individual through a fitness function. Based on the evaluation, decide which individuals will survive, reproduce or be killed. Apply genetic operations to the individuals selected for reproduction. If the criterion of stopping the process is not met, return to step 2. The criteria for stopping the evolutionary process can be when a maximum number of evaluations made, a maximum number of generations evolved or the fitness reaching a certain level or other measurable criteria given. When the genetic process is finished in GP, there will exist a program that will solve the problem. GP Theory The formal description of the GP functionality presented here is derived from Holland s formal description of Genetic Algorithms (Holland, 1975). An adaptive evolutionary system, as GP, always operates in an environment, E. It is only though the interaction with the environment that we can measure the performance of the learning system. This is a key factor in GP where the genetic process evolves new individuals, based on their performance. GP represents its individuals as a finite set of structures, S={s 1, s 2,, s n }, often referred to as genotypes and in GP, represented as source code individuals. An structure, s x, consists of a function set, F={f 1, f 2,, f m }, and a terminal set T={t 1, t 2,, t n }. Note that these could be the same terminal and function sets as in CxBR. The first step in each generation, to evolve better individuals, is to transform the genotypes into phenotypes, P={p 1, p 2,, p m }. This is a requirement for interaction with the environment. This is done with a genotypephenotype mapping function π such that: p x =π(s x ) In GP, the mapping function π is the compilation or interpretation of the source code, i.e., the genotype individuals. The interaction of the phenotype in the environment is in GP the execution of the phenotype: ϕ E (p x ) Next step in the GP process is to evaluate the performance of the individual through the fitness function µ E : µ E (s x, ϕ E (p x )) Note that the fitness function µ E is a function of both the performance of the phenotype in the environment and the genotype, s x. This could, as an example, be to encourage smaller individuals with unnecessary or less complicated source code. The fitness function returns a fitness measure based on how good this individual s combined performance was. Depending on the fitness measure of the individual, the selection scheme, α, selects an operator, ω Ω. Ω is the set of all operators where: Ω={ω x :S m S n } An operator ω maps m structures into a set of n possible modified structures. Individuals with a better fitness measure have a greater chance to be selected. Each operator type is a stochastic operator that will be triggered with a predefined probability. If neither stochastic operator indicates applicability, the selected individual will survive to the next generation without any modifications. After the selection scheme has selected individuals to breed and appropriate operators have been applied, a new population now exists, partly with new individuals and partly with individuals from the last population.

5 GP has the ability to evolve and develop command sequences (Koza, 1992). Besides the ability to build the knowledge elements in the context base, GP also has the ability to develop and destroy contexts and sub-contexts. This means that the GP possesses the feature to evolve appropriate sets of contexts within the context base. Since GP uses the same function and terminal sets as CxBR, GP could be used to incorporate knowledge in any context level or in any instances within CxBR where intelligent behavior is encoded. This means that we could choose to implement learning in any specific part of CxBR and construct the knowledge. Towards automation To be able to build models automatically, CxBR needs to be equipped with learning capabilities. Intelligent behavior, within CxBR, can be categorized in two groups, as shown above, actions and sentinel rules. At different Context levels, the sentinel rules determine which context will be active for that specific level. The structure of these sentinel rules are similar in all contexts at all levels. Each context has its own set of sentinel rules that determine if this context still shall be active or if it shall turn over the control to another context at the same level. This can be viewed as state transition rules where each state, i.e., context, has its own transition table. At the Main Context level, the action set tends to be more a collection of Subcontexts and less of other functions and terminals. At lower context levels the action set is less composed of Subcontexts. At the lowest context level, there are no Subcontext calls. Experience has shown that three, or at most four, levels of contexts, including the Mission Context, are normally sufficient. Individuals GP Fitness Simulator CxBR Figure 2. Learning by Observation: CxBR+GP From the discussion above, we can conclude that if we want to incorporate learning into CxBR, the learning - Recorded Human Data paradigm must be able to learn the applicable behavior in a specific context, i.e., actions, and also the appropriate context switches, i.e., sentinel rules. An extra learning feature would be if the system were able to optimize the number of appropriate contexts. This infers that the learning system shall be able to create new contexts if it seems to be appropriate, and also be able to prune less useful contexts. A learning paradigm that has those features is Genetic Programming (GP). Instead of creating the contexts by hand, we use the GP process to build the contexts. The GP s evolutionary process provides the CxBR frame with appropriate contexts and rules. The individuals in the genetic population are context parts and a simulator is used to simulate the model s behavior. The behavior from the simulator is then compared with the human performance and a fitness measure is established that evaluates the models appropriateness. See figure 2. The evolutionary process will strive to minimize the discrepancies between the performances of the contexts created by GP and the human performance in the simulator. The features of CxBR and GP show that the combination of CxBR and GP could be a feasible approach to learning by observation. By this combination, a system could be constructed that, by observing a human in a simulator, builds a context base for simulated entities with human behaviors. A positive feature of using GP as the learning algorithm is that it preserves many of the features of CxBR that makes it an appropriate paradigm to model human behavior. The tools GP uses to store the knowledge learned is the same tools as a programmer or knowledge engineer would have used developing the knowledge base, i.e., source code statements. This implies that the knowledge is easy to interpret if we want to include communicative features in the agents. The agents are able to express its action in a way that is easy to understand. There is very easy to interpret source code and convert it to written language. GP can also serve to later refine the learned knowledge. A simulated entity that has learned human behavior knowledge via observation can later continue to learn by experience. The same paradigm is then used, but instead of comparing its action to the recorded human performance, the fitness function evaluates its own performance and will hereby learn from experience. This is also a human feature that an intelligent simulated entity can and should incorporate. Implementation Strategy An analysis of the behavior structure in CxBR shows that the sentinel rules that control which context that shall be active is a classifying or clustering issue. The objective for the sentinel rules is to classify the current situation to an applicable context. When we look at the intelligence stored within the contexts the aim is to match the actions of the

6 model to the actions of the human. This tends to be more of regression analyzes to minimize the discrepancies between the model and the human performance. This is especially true further down in the hierarchy where the Sub-Contexts approaches low level behavior as motor skills. If this is looked upon from a machine learning perspective the learning tasks in CxBR would incorporate different types of learning strategies. GP is a generic approach that is capable of learning both types of strategies with none, or minor adjustments opposed to many other machine learning algorithms. The task of implementing a learning strategy that is applicable in all instances in the hierarchy of CxBR is very complex. The problem domain ranges from very low-level decision-making to high-level tactical decision-making. Many of the parts that constitute intelligent behavior are also correlated to each other. The sentinel rules that control the context switch and the action part in the different contexts could be correlated, i.e., the learning of these two instances could not be performed mutually exclusively. However, we can identify two main problems concerning the learning implementation: Hierarchical complexity Interdependency among contexts and context parts The first problem needs some sort of structure to the implementation of the learning in the different parts. The construction of the fitness function for a complex problem like that would also be very hard to implement. The fitness function is the part of the learning system that determines how well the system performs compared to the observed human. The search space for such complex problem would be huge. Different research approaches exist to attack this problem. One approach is to reduce the search space by letting the GP evolve higher-level behavior by using a meta-language where the function set is a set of low-level behavior routines (Luke, 1998). This restricts the problem solving in many ways and also contradicts the idea of learning by observation. We need to manually construct the behavior in the lower context levels that we do not try to learn. This means that there is a major part of the CxBR structure that needs to be manually created. Another approach would be to construct a very complex and sophisticated fitness function to control the learning in the different part of CxBR. This moves the complexity and manual design from the low-level behavior to the fitness function and the learning task is much more complicated. This still implies that a lot of manual coding needs to be conducted. Hsu and Gustafson (2001) propose a GP-approach where the high-level behavior would be broken down to lower level behavior that is first learned and then used at the learning of the higher-level behavior. The approach is called Layered Learning GP (LLGP) and it breaks up the complex problem into a hierarchy of sub-problems. Hsu and Gustafson showed promising results using the LLGP strategy. This is a type of bottom-up strategy and it suits our needs, since the problem domain in CxBR already has a hierarchical structure. To apply LLGP to implement learning by observation within CxBR, simply start the learning by looking at the lowest context level. The learning problems at this level are fairly simple and well defined. When the learning system has been able to establish the contexts at the lowest level the learning can continue with the next higher context level and the already learned context could be used as building structures in the GP s function set. The other problem is the issue of interdependency between the knowledge in the same level contexts and also between the action part and the sentinel rules in the same context. This means that the knowledge in the different interdependent parts could not evolve separately. Since the performance in one part is dependent on the performance in the other part, these two parts needs to be evolved at the same time. The interdependency between different contexts at the same level might not be that strong, since only one context can be active at the same time, i.e., mutually exclusive. Rather, is the dependency in their combined performance including the action and the context switching, i.e., sentinel rules. Their joint performance needs to be evaluated in some way, i.e., their action in the different context and the appropriate context switching at the appropriate time. An approach to this problem is to use the Cooperative Co-Evolutionary strategy. In this strategy you can have different populations evolving solutions to sub-problems in parallel. When it comes to evaluation of their performance, i.e., calculation of the individual s fitness, to evaluate their joint effort the best individuals from the other populations are included to produce and measure their performance. Mendes, et al. (2001) used Cooperative Co-Evolution to successfully evolve fuzzy classification rules. They used GP to evolve the different classification rules and a simple evolutionary algorithm to evolve the membership function definitions. In a similar way we could let the different contexts action sets evolve in parallel with their sets of sentinel rules and combine all, or some, of them when the fitness value is calculated. Combining LLGP and Cooperative Co-Evolution, a strategy could be defined to evolve the greater part of the intelligent knowledge within a simulated agent with tactical human behavior. Problem Specific Issues Even if there is a well-defined strategy on how the learning is to be conducted, there are some issues that need to be attended to before learning can take place. The No Free Lunch theorem (Wolpert and Macready, 1995; 1997) concludes that if learning is to be successful, the learning

7 algorithm needs to be adjusted to the problem at hand. If nothing is known about the problem to be solved or nothing is incorporated to the learning algorithm, there is only a 50% probability that it will perform better than a random search. In GP, there are two main features that needs to be set up for the problem at hand. First, a fitness measure needs to be developed that can determine the level of performance from the tested GPindividual. In our case, the individual is a part of the intelligent knowledge structure in an agent, and this part needs to be examined by running the agent in an environment where this structure could play a role. The fitness function then evaluates the performance of the agent, and rewards the individual with a corresponding fitness measure. In our case, we need to compare the performance of the individual with the observed human s performance. The other main feature of GP that needs attention when setting up a learning environment for a specific problem is the function set. The function set needs to be reduced to a set appropriate for the problem at hand, but one that is still rich enough to be able to solve the problem. Conclusions This paper has shown the advantages the new theory could achieve by combing CxBR with GP, i.e., learning by observation. The theoretical foundation of combining Genetic Programming and Context-Based reasoning has been presented. We have reached a point where the time consuming task of gathering, evaluating and implementing knowledge into our AI systems could be replaced by automatic modeling. The computational power and the research are closing the gap to fully automated human behavior modeling. The fundamental issue is that an automatic modeling engine shall replace the manual modeling, i.e. software development. If manual modeling were to find the best way of modeling tactical human behavior, we must not throw away these achievements. The most obvious approach for an automatic modeling engine would be to adopt the manual behavior. GP exhibits those features that could mimic the manual modeling process by addressing the issue of automatic programming (Koza, 1992). Using GP as the learning engine to a CxBR framework ensures that the model produced will inherit human features. Hence, GP will naturally extend CxBR with learning capabilities. The next step in the research is to begin experiments to show the practical feasibility of the theory. Initial experiments show encouraging results. References Abravanel, E., Ferguson, S. (1998) Observational learning and the use of retrieval information during the second and third years Journal of Genetic Psychology; Dec 1998, v.159, 4, 455(2) Bass, E. J., Zenyuh, J.P., Small, R.L. and Fortin, S.T. (1996). A Context-based Approach to Training Situation Awareness, Proceedings of the Third Annual Symposium on Human Interaction with Complex Systems, Los Alamitos, CA: IEEE Computer Society Press, pp Brezillon, P., Pasquier, L. and Pomeroi, J-C., Reasoning with Contextual Graphs, European Journal of Operations Research, Deutsch, S. (1993) Notes Taken on the Quest for Modeling Skilled Human Behavior Proceedings of the Third Conference on Computer Generated Forces and Behavioral Representation, Orlando, FL, March 17-19, pp Gonzalez A. J. and Ahlers, R. H. (1994) "A Novel Paradigm for Representing Tactical Knowledge in Intelligent Simulated Opponents" Proceeding of the IAE/AIE Conference, May 31-June 3, 1994, Austin, Texas. Gonzalez A. J. and Ahlers, R. H. (1998) "Context-Based Representation of Intelligent Behavior in Training Simulations Transactions of the Society for Computer Simulation International, volume 15, no 4, December 1998 Gonzalez A. J. Georgiopoulos M. DeMara R. F. Henninger A. Gerber W. (1998) "Automating the CGF Model Dvelopment and Refinement Process by Observing Expert Behavior in a Simulation" Proceedings of the Computer Generated Forces Conference, Orlando, 1998 Henninger A., Gonzalez A., Gerber W. Georgiopoulos M., DeMara R. (2000) On the Fidelity of SAFs: Can Performance Data Help? Proceedings of the Interservice/Industry Simulation and Education Conference, Orlando, FL, Holland, J.H. (1975) "Adaptation in Natural and Artificial Systems", University of Michigan Press, Reprinted by MIT Press, 1992 Hsu, W.H., Gustafson, S.M. Genetic Programming for Leyered Learning of Multi-agent Taks, Proceedings of the Genetic Evolutionary Computation Conference, GECCO 2001, San Francisco, CA, July 9-11, 2001 Koza, J. R. (1992) "Genetic Programming", MIT Press, 1992, ISBN Luke, S. (1998) Genetic Programming Produced Competitive Soccer Softbot Teams for RoboCup-97, Proceedings of the Third Annual Genetic Programming

8 Conference, p , Morgan Kaufmann, Los Altos, CA, 1998 Mendes, R.F. Voznika, F.B. Nievola, J.C. Freitas, A.A. Discovering Fuzzy Classification Rules with Genetic Programming and Co-Evolution, Proceedings of the Third Annual Genetic Programming Conference, p , Morgan Kaufmann, Los Altos, CA, 1998 Schaal, S., (1999) Is imitation learning the route to humanoid robots? Trends in Cognitive Sciences, 3(6), Turner, R.M. (1993) Context-sensitive reasoning for autonomous agents and cooperative distributed problem solving, Proceedings of the 1993 IJCAI Workshop on Using Knowledge in Its Context, Chambéry, France Turner, R.M. (1999). A Model of Explicit Context Representation and Use for Intelligent Agents, Springer- Verlag, 1999 van Lent, M. and Laird, J. (1998). Learning by Observation in a Tactical Air Combat Domain, In Proceedings of the Eighth Conference on Computer Generated Forces and Behavior Representation. Orlando, FL., May, 1998 Wolpert D.H. and Macready, W.G. (1995) No free lunch theorems for search, Technical Report SFI-TR , Wolpert D.H. and Macready, W.G. (1997) No free lunch theorems for optimization. IEEE Transactions on Evolutionary Computation, Vol. 1, No. 1, pp , 1997