An early Agent-Based stock market: replication and participation

Transcription

1 An early Agent-Based stock market: replication and participation László Gulyás 1, Adamcsek Balázs 2 and Árpád Kiss2,3 1 Computer and Automation Research Institute Hungarian Academy of Sciences Kende u Budapest, Hungary gulyas@sztaki.hu 2 AITIA Inc. Infopark sétány Budapest, Hungary {abalazs, akiss}@aitia.ai 3 Loránd Eötvös University Pázmány Péter sétány 1/c 1117 Budapest, Hungary Abstract. The Santa Fe Artificial Stock Market (SFI-ASM) is one of the most prominent models of agent-based finance, a computational approach to study the complex system of the financial market. The SFI- ASM model has two regimes: in one of them the simulated time series data is consistent with the rational expectations equilibrium, while in the other, simulation results appear to be in accordance with actual financial time series data. The goal of this paper is twofold. First, it reports on the results of porting an early version of the SFI-ASM model onto the RePast simulation platform. Replications form a very important methodological step in order to scrutinize computational results. Second, the paper describes an extension to the model that takes it from the realm of agent-based theoretical experiments to that of participatory simulation. In participatory simulations some agents are artificial, while human subjects control others. This setup offers a great opportunity to test both the assumptions and the results of the model. The experiences of the first set of participatory experiments are also discussed, demonstrating how technical trading may lead to market bubbles. Keywords. Artificial stock market, Agent-Based models, simulation, artificial and human agents, participatory experiments, technical trading, market bubbles. J.E.L. classification: C63, G19. M.S.C. classification: 81T80, 91B26, 92B20. 1 Introduction Economics provides many examples of social systems involving complex interactions among many individuals. Traditional models of these systems seek to

2 48 László Gulyás, Adamcsek Balázs and Árpád Kiss simplify human behavior and yet to derive easily characterized aggregate macro features. In some cases, for example, in case of many finance models, however, the approach has only yielded mixed success. This led to the emergence of the novel approach of agent-based finance. The most prominent model of which is the Santa Fe Artificial Stock Market (SFI-ASM), where artificial agents make investment decisions. The SFI-ASM is a model with a risk-free financial asset (e.g., Treasury bills) available in infinite supply that pays a constant risk-free return rate per period, and with a single risky stock, whose fundamental share value is unknown to the traders. Traders are identical except that each trader individually forms his trading rules over time through an inductive learning process. Each trader chooses his portfolio of financial assets in each period in an attempt to maximize his wealth. At the start of the market process, each trader has a set of rules that it evolves over time in such a way that new rules are continually being introduced. This setup yields a stable system with two distinct behavioral regimes. [8] In simple cases, the simulated time series data is consistent with a rational expectations equilibrium. In contrast, in more complex setups, the market does not appear to settle down to any recognizable equilibrium. While market price vaguely follows the fundamental value of the stock, upward and downward deviations exist that may be called crashes and bubbles. In this regime, simulation results appear to be in accordance with actual financial time series data. [1] [8] This paper first reports on the results of porting the SFI-ASM model onto the RePast simulation platform. Replication is a very important methodological step in order to scrutinize computational results. Our implementation confirms the results in [8]. Furthermore, we describe an extension to the model that takes it from the realm of agent-based theoretical experiments to that of participatory simulation. Participatory simulation is, in effect, a bridge between the laboratory experiments performed in experimental economics and the abstract explorations carried out in agent-based modeling. In a participatory simulation some agents are artificial, while human subjects control others. This setup offers a great opportunity to test both the assumptions and the results of the model. For example, participatory experiments may help calibrating certain parameters. On the other hand, it may help ascertaining the stability of the outcomes, e.g., when the artificial agents are confronted with agents with utterly different behavioral patterns. Also, participatory simulation allows testing certain hypotheses about human behavior. Our first experimental results contribute to the latter two topics. The results show that even a few agents that play a different strategy from that published in the original paper may significantly alter aggregate market performance. Furthermore, the experiments also point out differences in human and computational agent behavior in this particular setting, and demonstrate the effects of technical trading. The paper is structured as follows. Section 2 describes the SFI-ASM model. This is followed by a summary of our replication efforts. Section 4 introduces the Participatory Santa Fe Artificial Stock Market (PSFI-ASM) model. Section

3 An early Agent-Based stock market: replication and participation 49 5 describes the settings of our first experiments with the PSFI-ASM model and reports on the results obtained. Section 6 concludes the paper. 2 The Santa Fe Artificial Stock Market model In the SFI-ASM model, time is broken up into discrete time periods. There exists a risk-free financial asset (e.g., Treasury bills) available in infinite supply that pays a constant risk-free return rate per period. There also exist a risky stock, whose dividend is generated by an exogenous stochastic process. Computational agents, the traders, attempt to maximize their wealth by choosing their portfolio in each time period. There are no complex instruments such as options, nor is direct interaction between the agents. Traders are identical except that each trader individually forms his trading rules over time through an inductive learning process. At the start of the market process, each trader has a set of randomly generated rules. Each rule determines what the agents should do in a given market situation. The possible actions are buying or selling a stock, or doing nothing. Several versions of the SFI-ASM model exist. More modern versions are improved, among others, in their economic realism and in the use of more sophisticated trading rules. Keeping our sight on our final goal of transforming the model into a participatory agent-based simulation, we trade-off sophistication for simplicity. Therefore, in the following we will only consider the early version of the model published in [8]. 2.1 The artificial stock market Let t = 0,1,2,... identify time periods, t = 0 corresponding to the initial state of the system. Moreover, let A = {a 1,a 2,...,a N } be the set of agents, the positive integer N denoting the number of agents in the system. The number of stocks held by agent a in the t th time period is given by h t a R, R standing for the set of real numbers. Similarly, m t a R denotes the amount of financial asset (money) the agent owns. Notice, that both holdings and money is represented as a real value, for reasons of simplicity discussed further below. The price p t R per share of the stock depends on the overall buying and selling behavior of the agents. The stock, however, may also pay a dividend d t R in money. The agents money is assumed to be invested in a fixed-rate fund, such as a savings account, that pays an interest rate r in each period. Agents can thus make profit in three ways: due to the interest derived from their cash, due to the dividend stream, or through speculation on price changes of their shares. The wealth w t a R of agent a is defined as w t a = m t a + p t h t a. (1) At the beginning of the t th period, the interest and the dividend is paid, so that M t+1 a = r m t a + d t h t a, (2)

4 50 László Gulyás, Adamcsek Balázs and Árpád Kiss for each a A and t 0. Then agents have the chance to change their holdings. They can sell or buy one share, subject to the availability of shares or money, respectively. The latter constraint can be relaxed by allowing the agents to borrow money subject to a certain limit. At the end of the period, bids and offers are matched and the market clears. As there must be a buyer for every seller, agents may not be able to achieve the holdings they desire. Let b t a = 1, if agent a attempts to buy a share in the t th period, and b t a = 0 otherwise. Similarly, let o t a = 1, if agent a offers to sell, and o t a = 0 otherwise. Furthermore, let B t and O t stand for the totals of bids and offers, respectively. B t = a A b t a (3) O t = a A o t a (4) If B t = O t, then all bids and all offers are satisfied, giving h t a = h t 1 a + b t a o t a (5) for each a A and for each t > 0 (where either b t a or o t a is zero). However, if B t > O t then all offers are fully satisfied, while only a fraction O t /B t of each bid is filled, giving h t a = h t 1 a Similarly, the case when B t < O t yields 4 h t a = h t 1 a + Ot B t bt a o t a. (6) + b t a Bt O t ot a. (7) The volume of trade V t in the t th period is then defined as V t = min(b t,o t ). Notice that this rationing scheme may result in non-integer holdings. To complete our description of the market, we need to define how the dividends d t and prices p t are set. Dividends are generated by a discrete, stochastic colored noise process, d t+1 = d + ρ(d t d) + ξ t, (8) where d is the theoretical dividend mean, ρ is a speed parameter, and ξ t is a Gaussian noise source with mean 0 and a variance δ 2. This is the discrete version of the mean-reverting autoregressive Ohrnstein-Uhlenbeck process (see [8] and [3]), often used as a simple model of stock market time series. [3] [1] In contrast to the purely stochastic series of dividends, prices depend on the actual bids and offers. If more agents want to buy than sell, the price should go up, while it 4 The rationing scheme described here is far from being fully satisfactory. In fact, this is one point where more modern versions of the SFI-ASM had been significantly improved. However, for the one-stock scenario discussed in this paper it works well, even despite its lack of realism.

5 An early Agent-Based stock market: replication and participation 51 should fall, if the supply exceeds the demand. This is achieved by the following formula. p t = p t 1 (1 + η(b t O t )) (9) The parameter η has a crucial role in determining market behavior. When it s small, the price adjusts slowly to different market conditions. On the other hand, a large η leads to large oscillations of price. In principle, it is assumed to be small enough to ensure that η(b t O t ) 1. Obviously, agents have to pay for filled bids and they cash in on satisfied offers. This yields the following formula for agent a s money at the end of period t, which completes the specification of the market. 2.2 The trading agents m t a = M t a V t B t bt a p t + V t O t ot a p t (10) The trading agents of the SFI-ASM model try to maximize their wealth by regularly changing their portfolio. Their behavior is based on rules that specify what to do when certain market conditions are met. The general form of the rules is as follows: (condition, action, strength). The third element of the triple is a real value, whose role will be discussed later. The action is a simple ternary choice: (i) bid: b t a = 1,o t a = 0, (ii) offer: b t a = 0,o t a = 1, or (iii) neither/hold: b t a = o t a = 0 (default action). The condition part of a rule is a fixed-length string of symbols drawn form the alphabet {0, 1, *}, e.g., 11*0*****1*0. This string is matched against a binary string (with 0s or 1s only) of the same length, representing the current state of the market. 0s and 1s in the condition string only match to the same symbols in the market string, while *s match to any value. The symbols represent market indicators, such as the price is above the fundamental value, or the price is higher than the 100-period moving average, etc. When the corresponding statement is true, the appropriate symbol is 1 and 0 otherwise. Therefore, rules specify certain actions, based on the state of market indicators, allowing for some of them to be indifferent to the application of the given rule. Agents have a set of such rules. Each time the agent has to make a decision, it first lists those rules whose condition is met and whose strength is positive. Next it selects one of these randomly, with probability proportional to strength. The action of this selected rule is then executed. If there is no matched rule, the agent defaults to the neither/hold action. At the end of the period, the strength of each rule whose condition was met is updated, according to the following formula. s t+1 a,k = (1 c) st a,k + A a,k c π t (11)

6 52 László Gulyás, Adamcsek Balázs and Árpád Kiss where s t a,k is the strength of agent a s kth rule in the t th time period and c is a small constant. A a,k is a numerical representation of the rule s action, so that 1, if the action is sell A a,k = +1, if the action is bid. (12) 0, otherwise π t is the net profit made by investing in one share of stock in the t th period. Its value is given by π t = p t (1 + r)p t 1 + d t. (13) The structure described above is a classifier system. [6] Its rules classify the states of the market into categories, and then provide probabilities for each possible action to be taken in each category. The agents in the SFI-ASM model, however, may also improve upon their rules. This is done by the application of a genetic algorithm [6] [7], which is executed at random intervals. When the algorithm is run, the agent replaces 10-20% of its weakest rules by new ones. The new rules are copies of some of the strongest ones, selecting candidates with a probability proportional to the rule s strength. However, the copied rules are modified by mutation or crossover. Mutation randomly changes bits of the rule, with probabilities adjusted so that the average number of *s stays constant. Crossover combines a pair of parent rules, getting a part of the new rule from one parent, and the rest from the other. 3 Replication results Testing published results is a most important step in the progress of science. However, computational results don t yield easily to tests. This is because the implementation of the simulation may obscure important details, assumptions or even mistakes. In addition, the code is poorly suited for publication in research papers, the traditional media of science. Therefore, computational results may go untested for longer periods of time. This emphasizes the importance of replication, the vehicle of testing for computer experiments. Replication means the re-implementation of the simulation by independent authors, based on the published specification of the model. This section reports on our efforts to replicate, in RePast [10], the version of the SFI-ASM model that was described in the previous section. The original version of the SFI-ASM model was implemented in C. [6] [7] It was later replicated in Swarm and in RePast. For a description of these systems the interested reader is referred to the websites [11] and [10], respectively. The replication in RePast provides a good example of the significance of replication. During this replication effort, Norman Ehrentreich has discovered a flaw in the mutation operator in the agents genetic algorithm that corrupted part of the results. [3] However, no replication of the earlier model published in [8] was made so far. Another motivation to choose the early version was its better applicability to participatory simulations.

7 An early Agent-Based stock market: replication and participation The early artificial stock market model The RePast implementation 5 of the early SFI-ASM model closely follows the specification of Section 2. In the following further details are given with respect to parameter values and other considerations. These are based on information in [6], [7], [8] and [3]. The stochastic process generating the dividends is slightly different from the one given in Equation (8). It takes the following form. d t+1 = max( d + ρ(d t d) + ξ t, ), (14) In effect, this ensures that d t > 0, for each t 0. The values of the parameters are ρ = 0.95 and d = Agents are initially endowed with certain money m init and shares h init. In the simulation runs discussed here, agents start with m init = 200 units of money and h init = 5 shares of stock. Agents are allowed to borrow money (to buy stocks) to the limit of their initial monetary endowment. That is, they are allowed to buy until m t a m init holds. The interest rate on this credit r d is equal to the interest rate paid for the fixed-rate investments, i.e., r d = r = Agents have 60 rules whose conditions reflect on the 12 market indicators shown on Table 1. Bits 11 and 12 are zero information bits, providing a way to check whether the agents behavior is actually dependent on market processes. The table also shows the possibility of rules whose conditions can never get matched, because they contain contradictory conditions. For example, a string in which bit #1 is 0 and bit #2 is 1 would imply that 1/2 < p t r d t 1/4, a clear contradiction. For this reason, agents have a meta-rule that generalizes rules that haven t been matched for a long period of time. Generalization is done by randomly replacing a few bits in the condition string with the * symbol. The length of the maximum sleeping period for rules is 200 in our studies. Table 1. Market Indicator Bits (based on [6]) Bit Market Indicator 1 p t r d t > 1/4 2 p t r d t > 1/2 3 p t r d t > 3/4 4 p t r d t > 7/8 5 p t r d t > 1 6 p t r d t > 9/8 7 p t > 5-period moving average price 8 p t > 10-period moving average price 9 p t > 100-period moving average price 10 p t > 500-period moving average price 11 On: 1 12 Off: 0 5 Available from the authors upon request.

8 54 László Gulyás, Adamcsek Balázs and Árpád Kiss Agents have another meta-rule to help optimizing their strategies. According to this, the action of rules with negative strength is reversed. The intuition behind this is that if a condition was particularly weak for buying, it should be good for selling, and vice versa. What is left to specify, is the settings for the genetic algorithm (GA). The number of rules replaced each time the GA runs is 10 in our experiments. The c parameter from Equation (11) is For new rules, the probability of crossover is 0.3. In this case the condition strings of the two parents are crossed once, at a random point, while the action is the action of a randomly chosen parent. The strength of the crossed-over rule is reset to 0. In case of mutation, a few randomly chosen bits are changed. The probabilities of going from 1 to 0, 0 to *, or * to 1 have to be adjusted, however, to ensure that the average number of *s remains constant. As pointed out by Ehrentreich [3], the model s original implementation was flawed at this point. Therefore, our implementation follows the modified algorithm discussed in [3] to calculate probabilities. 3.2 The results of replication The main findings of the early SFI-ASM model, reported in [8], can be summarized as follows. In simple cases when the agent population is small, or they only use a few rules, or facing a low-variance dividend stream, the agents converge to an equilibrium, in which price follows fundamental value, volume is low, and there are no significant anomalies, such as bubbles or crashes. The agents become relatively homogeneous, using only a few simple rules. On the other hand, in more complex environments, there is no clear equilibrium. Although, the price usually stays close to fundamental value, it also displays major upward and downward deviations that may be regarded as bubbles or crashes. The agents become heterogeneous, and thus trading volume also remains relatively high. 6 Our RePast implementation confirms these findings. Fig. 1 shows a typical run in a complex environment. Deviations from fundamental value are visible and trading volume keeps oscillating around 10% of the supply. The latter suggests that agents are heterogeneous. High volume V implies that both demand and supply is high, both being at least V. Therefore, the agents rule sets must differ significantly, since facing the same market conditions V of them think it best to buy, while V of them to sell. Indeed, Fig. 2 shows statistics of the fraction of used (non-*) bits in the agents rule sets. Clearly, some agents have very specific rule sets (the fraction being close to 1), while others apply very general rules that only depend on a few indicators. 6 This is in contrast to later versions of the SFI-ASM model (e.g., in [1]). Despite that they also have two regimes: equilibrium is reached when the agents learning rate is slow, while market-like deviations arise when they learn fast. These results were later questioned in [3]. It was found that after fixing a technical problem, agents become similar and price follows fundamental value.

9 An early Agent-Based stock market: replication and participation 55 a) b) Fig. 1. Simulated time series of a) price (smooth curve) and fundamental value (crossed line) and of b) trading volume. The data is typical of a 100-agent market with fast learning rate (GA-activation in every 3 rd period) a) b) Fig. 2. Statistics of the fraction of used (non-*) bits in the agents rule sets. a) The minimum (crossed line), maximum (rectangles) found in agents, and the average (smooth line) versus time. b) The distribution of the same measure among the agents

10 56 László Gulyás, Adamcsek Balázs and Árpád Kiss Another measure of the agents homogeneity is the distribution of their wealth. Initially, every agent is endowed with the same wealth. In contrast, Fig. 3 shows significant heterogeneity after a certain period of time. Fig. 3. Wealth distribution in the middle of a typical run. The vertical axis shows the number of agents in the given tenth (shown on the horizontal axis between the minimum and maximum wealth) The above results suggest that, in complex environments, the agents of our implementation grow heterogeneous, both in their wealth and in their rules. At the same time, as a community, they learn to manipulate price in such a way that it follows fundamental value subject to a certain range of error. The level of error appears to be dependent on the frequency the genetic algorithm is run. If learning is slow, the deviations are larger, at least in the beginning. This is in contrast with [1], but not with [8]. Despite the agents apparent ability to follow the fundamental value, their heterogeneity, especially that of their wealth, suggests that interesting casting lies behind this social learning phenomena. Some agents grew smart and become wealthy, while others learn to be dumb, and loose out. In fact, this is an obvious consequence of the closed system, but interesting nonetheless. In their conclusions, Parker et. al. emphasize the self-organizing aspects of the SFI-ASM [8], pointing at the sellers and buyers that mutually adapt their behavior, and display organized system-level performance. In light of our replication studies, this co-evolution appears to mean that some agents, in effect, have learned to sacrifice their wealth.

11 An early Agent-Based stock market: replication and participation 57 4 A participatory stock market model The last section summarized our efforts to replicate the results of an early version of the SFI-ASM model. It was shown that our implementation yields results similar to those reported in [8]. More importantly, the behavior of the artificial stock market was, in many respects, similar to that of real world markets. In the simulated time series, price followed fundamental value with occasional deviations. Arguably, this was due to the co-evolution of the agents strategies. That is, the agents managed to adapt to one-another, i.e., they coordinate despite their heterogeneity. While significant, these findings leave open the question of whether these agents would be able to adapt to any strategy in the same way as they did to those of their fellow agents. In other words, it is unclear how well their meta-level rules would perform in a less sterile, more open environment. The same time, it would be interesting to know how the artificial stock market would appear to humans from the inside. Despite its simplicity, the aggregate behavior of the system is quite complex, lending itself to comparison with complex real world systems. Less obvious, however, is whether humans would perceive it as such a complex system. In short, we are interested in what effect human players would have on the system, and conversely in how the system would affect them. Conducting theoretical experiments with human participants is not new to the practice of science. It s not unheard of in economics either. Experimental economics has a long tradition, it s growing literature dates as far back as the early 20 th century. [5] It s often been applied to test the effect of human cognition to economic behavior, learning and adaptation. [2] Moreover, it has helped forming theories about coordination and the failure of it, e.g., in social traps. [9] A prominent application area of experimental economics has always been trading, with special emphasis on asset markets. Traditionally, economic experiments are carried out in a laboratory setting, participants playing a game according to a strictly defined protocol. Experimenters may use different techniques to record the unfolding of the game and to extract results from the recorded data. In the past, typical methods were observation and videotaping, and also questionnaires. Quite naturally, computers began to play an increasing role recently. The advent of the Internet opened up another path, broadening the scope of some experiments both in space and in time. It is safe to say that at least a handful of them is carried out somewhere at any particular moment in time. For example, at the time of this writing, a web-based artificial stock market game is going on at [12] [13] On the other hand, participatory simulation adds an interesting twist to the general approach of experimental economics. In these experiments a number artificial agents also participate in the game. Augmenting the studied population with programmed actors may help generating particular scenarios for the human participants. Also, it can be applied as a means to generate crowd behavior. Furthermore, participatory simulation can be used to achieve a different end. The introduction of human participants may help testing the sensitivity and

12 58 László Gulyás, Adamcsek Balázs and Árpád Kiss assumptions of a computer simulation. It is often helpful to be able to compare computational thought experiments with the behavior of actual people. This is the motivation behind the Participatory SFI-ASM model described in the next section. Agent-based modeling is especially suitable to this task as its inherent representation precisely defines the interfaces of the individual actors, together with the allowed interactions both among them and with their environment. This makes it conceptually straightforward to include agents, whose behavior is defined externally by a human participant. It is important to emphasize that due to the well-defined interface of the actors, artificial and human agents are undistinguishable from the point of view of the model, or from that of the participating agents. 4.1 The participatory model This section introduces the Participatory SFI-ASM model (PSFI-ASM), the participatory extension of the early SFI-ASM model described earlier in this paper. This model and the experiments described later are the first application of our General-Purpose Participatory Architecture developed for the RePast agentbased simulation platform. The GPPAR package is a collection of Java classes [4] that helps transforming RePast models to participatory simulation. It s applicable to a wide range of models, even to those that were written without the knowledge of GPPAR. A more detailed technical description of GPPAR will be reported elsewhere. 7 In GPPAR, the simulation runs on a central server. Artificial agents inhabit the server, while human agents connect to the simulation via the network by running a client application on their own computer. The connections and all communication are handled by the GPPAR infrastructure. Therefore, artificial and human agents are undistinguishable from the point of view of the model. This is illustrated by the fact that when clients log in, technically, they replace an artificial agent in the model s pool. During the design of GPPAR, special emphasis was given to the ability to replay experimental runs. This was achieved by an extensive logging functionality that keeps track of the actions of both the artificial and human agents. See Fig. 4 for an illustration. This was augmented by a special execution mode of GPPAR, in which the simulation is replayed, based on a given log file. This way, the unfolding of events can repeatedly be observed, and results are easy to analyze. In the SFI-ASM model, agents interact by indirect means only. Their only action is attempting to buy or sell a share of stock, or to remain inactive. Consequently, human participants are presented with an interface where they can press any of three buttons, corresponding to the above choices. To make their choices, agents have access to various pieces of information. They are obviously aware of the market indicators (see Table 1). Also, they must be aware of their 7 The GPPAR package is available from the authors upon request.

13 An early Agent-Based stock market: replication and participation 59 ******************* New Turn ****************** StockPrice= StockDividend= Artificial Action, 43:null*** wealth= Artificial Action, 73:bid*** wealth= Artificial Action, 17:null*** wealth= Artificial Action, 16:offer*** wealth= Participant Action, 0:null*** wealth=575.0 Artificial Action, 38:bid*** wealth= Artificial Action, 11:null*** wealth= Artificial Action, 85:offer*** wealth= Participant Action, 1:offer*** wealth= Artificial Action, 64:bid*** wealth= Artificial Action, 46:null*** wealth= Artificial Action, 28:bid*** wealth= Participant Action, 2:bid*** wealth= Artificial Action, 20:bid*** wealth= Fig. 4. Log file extract (Participatory SFI-ASM simulation) own assets: their money and their shares. While the latter doesn t follow immediately from the model description in Section 2, it is implicit in the limitations given by the budget constraints, i.e., that they cannot sell when they don t own any share and that they cannot spend limitlessly. For reasons of understandability, we have decided to provide human agents with the base values of the market indicators. That is, they are not confronted with a string of bits, but rather, they are given the values of measures (i.e., p t r d t and the moving averages). Also, the last two, zero information bits of the indicator string are kept from human participants. Following a similar argument, they are presented with their own money, number of shares and wealth. While artificial agents are not directly aware of these values, it would be unnatural to keep them from human players. 8 There is also a piece of implicit information available to programmed agents. Artificial agents update the strengths of their matched rules based on Equation (11). This includes the net profit made by investing in one share of stock in the given period as defined in Equation (13). Therefore, human players are also presented with this information. Also, they would hardly feel comfortable in the environment unless the running price of the stock is displayed to them. The user interface of the Participatory SFI-ASM client application is shown on Fig In fact, it would be interesting to see how the behavior of artificial agents would change if they were able to evolve strategies that take their money and share holdings, and their wealth into account. It seems reasonable to think that, for example, the level of risk-aversion should be different when being poor versus after growing wealthy.

14 60 László Gulyás, Adamcsek Balázs and Árpád Kiss Fig. 5. The Client Application of the Participatory SFI-ASM model Our goal, in creating the experimental environment for the human participants, is to present them with a game as real and as exciting as possible. Therefore, we want to imitate the fast-changing, on-line nature of stock markets, in which prices change by the fraction of a second. However, this presents a technical problem as the original SFI-ASM model is organized around rounds, each agent making a decision in each round. Clearly, human agents cannot be expected to make decisions in the fraction of a second. The GPPAR package offers several ways to deal with the extended time requirement of human participants. The simplest of which is a timing-out system, in which participants are given a certain amount of time to make their move. If they fail to do so, a default action is executed. However, giving enough time to deliberate would slow down the simulation significantly, destroying the illusion of a real-time game. Therefore, we decided to keep the time-out low (at the order of 0.1sec), but regard the participants bids and offers as continuous. That is, the last action is resubmitted in each round, until the player changes it, e.g., to do nothing. Technically, this is implemented by defining the default action, submitted when timing-out, as the last action initiated by the user. However, there is a more fundamental problem with the real-time version of the SFI-ASM model. According to the specification in Section 2, interest and dividend is paid in every round. This results in the continuous growth of wealth, which, given a high frequency of rounds, becomes a very rapid one. Besides being at odds with reality, this implies that the relative weight of the profit that can be made on stocks declines very fast. (This is because of the limited supply of shares, which is the total sum of the agents initial endowments.) To overcome this problem, we stretched out the growth process by rarefying interest and dividend payments to every 5 th period.

15 An early Agent-Based stock market: replication and participation 61 a) Price (smooth line) and fundamental b) Wealth (smooth line) and base value (crossed line) time series wealth (crossed line) time series Fig. 6. Motivational time series graphs provided for human agents In order to foster the illusion of a real-world stock market, we also provide human agents with time series graphs. First, they are able to see the history of the price and the fundamental value (i.e., d t /r). This is illustrated on the left panel of Fig. 6. Another reason for graphic presentation is human preference for visual information. Also, we want participants to be motivated to give their bests. Naturally, the goal set for them is to make a fortune. However, due to reasons discussed above, agents in the SFI-ASM increase their wealth, more or less independent of their actions. Therefore, we introduce the notion of base wealth, which stands for the current worth of the agents initial endowment. In other words, it is the wealth of an agent that did nothing, but kept its initial money in the bank, and its endowment of shares in the drawer. The second time series graph displayed by the PSFI-ASM client, shown on the right panel of Fig. 6, provides this information to the users. It shows the time series of the player s wealth, together with that of the base wealth. The current difference between these values is also shown numerically among the agent s statistics. This way, players can have a notion of their performance, both current and historical, which may motivate them to make an effort. Obviously, the additional information provided for human players (e.g., the history of values, the running price, etc.) creates asymmetry between the human and the artificial side. However, as argued above, the asymmetry is already inherent in the agents different computational ability. Thus, we consider the above setup to be a good and balanced compromise.

16 62 László Gulyás, Adamcsek Balázs and Árpád Kiss 5 Experimental results This section presents the results of our first experiments with the Participatory SFI-ASM model. One major goal of these experiments was to demonstrate the workability of the PSFI-ASM model. Besides this, however, we were seeking information about a number of exciting questions. For example, [1] and [8] claims that their classifier agents adapt to the different behaviors of the market, thus making sure price always stays relatively close to fundamental value. However, our hypothesis was that this is mainly due to their inherent similarity. That is, despite the significant differences in their parameters and rule sets, the meta-level algorithm of the agents remains the same. This fact could ease the process of adaptation, and could smooth out market deviations. Therefore, we expected that human participation would increase the deviation from the rational expectations equilibrium. If true, this would present itself in the form of larger crashes and bubbles, and perhaps in an increased level of trading volume. We were also interested in the performance (relative wealth) of the different types of agents. On one hand, computational agents apply a very simple learning algorithm. Despite the simplicity of their rules and their information set, the artificial agents are fast, so the best among them, in effect, optimize their behavior to the current and past states of the market. On the other hand, human participants face a large set of rapidly changing information and are forced to make quick decisions. Therefore, we hypothesized that computational agents would outperform human participants, at least initially. However, humans can carry over their knowledge to subsequent runs. Hence, we were interested in the rules the players would develop, and in the effects of potential learning in repeated games. Finally, more as a measure of our success of motivating our players, than a question of theoretical importance, we were interested in how human participants perceive the market. Will they think it is behaving in a complex fashion, or will they see through it easily? Will they be excited and start theorizing about their best strategy, or will they fall back to a simple, monotonic pattern of behavior? The answers to these questions will be discussed next. 5.1 The first participatory experiments The first set of experiments was carried out at Loránd Eötvös University, and at AITIA, Inc., both in Budapest, Hungary. The participants were computer science students, and employees of the company, respectively. They were all skilled computer users, but they lacked any stock market experience. The participants used personal computers (running a mix of operating systems, including MS Windows 2000 and Linux), connected through a local area network. Despite their physical proximity, participants were not allowed to discuss the game or their performance, before the end of the experiments. The only information the players received between the runs was the identity of the overall winner. The

17 An early Agent-Based stock market: replication and participation 63 PSFI-ASM simulation server was also running on a networked personal computer. The experiments began with an introductory session, where participants were told the rules and goals of the game, and were the client application s interface was explained to them. The description of the game rules was limited to the abstract interface of the SFI-ASM agents. That is, they were told about their possible actions, or about the fact that their bids would not necessarily be filled. However, they didn t know the exact rules of market clearing, nor did they have direct knowledge about how the price was determined. Similarly, they had no knowledge about the dividend stream, except that it was a dynamic process, following its internal logic. The introductory session was followed by a short trial run of the system, when the participants had a chance to see the system in action. Afterwards, they could ask questions, which were answered by the experimenters, unless they were about the internal details of the system, or were directly connected to one of the hypotheses of the experiments. Then the experiments were run. Each time, the simulation was stopped without warning, at the discretion of experiment leader who was uninformed about the actual course of the current run. The reason for this random stopping rule was to avoid human strategies that could take the extra information of the experiment s length into account. In the particular experiments, the system was run for about 5 minutes ( rounds). Finally, after the intended number of experiments, the participants were asked to fill in a questionnaire, which contained questions about their strategy, the use of the different market indicators, about their errors or possible improvements, and about their perception of the game. It also contained questions about the user interface, and about possible technical improvements. The results presented in the following section are based on runs with 8-10 human participants, facing 20, 100 or 400 artificial agents. 5.2 Results of the experiments This section summarizes the key findings of our first set of experiments with the PSFI-ASM model. We found, and this was backed by the questionnaires, that the players enjoyed the game, so we succeeded in constructing a challenging environment that motivates users. The players also confirmed our expectations about the preference for visual information. In fact, they were asking for more graphs, and, as one would expect, for more information. More importantly, however, we found that our first hypothesis was true. The presence of human traders yielded higher deviations. One measure of deviations is the cumulative difference between stock price and fundamental value. D t = t pi di r (15) i=1

18 64 László Gulyás, Adamcsek Balázs and Árpád Kiss The a)-c) graphs on Fig. 7 plot D t versus time, comparing participatory experiments (Dparticipatory t ) to corresponding simulations with identical parameters (Dcomputational t ). It is clear that significant human presence increases the level of deviation. The strength of the effect is less clear, however, since Equation (9) implies that larger populations (N) yield larger deviations, if η remains constant. To overcome this problem, η was set to a higher value for the small population (20 agents). Graph d) on Fig. 7 shows the normalized difference after 1050 steps as a function of the percentage of human participants. By normalized difference we mean the difference between Dparticipatory t and Dt computational, divided by the number of agents. The graph suggests that the more human players, the larger the deviation is. Unfortunately, we don t have enough data to test this hypothesis. Especially, as the graphs show the repeated performance of the same participants, in the order of b), c) and a). Therefore, the differences may also show the effect of human learning (this issue will be discussed later). a) 2% (8 players of 400 agents) b) 8% (8 players of 100 agents), second run c) 40 % (8 players of 20 agents) d) The scaling of the normalized difference as a function of the percentage of human participants Fig. 7. Cumulative market deviations in runs with and without human players. (The deviation axis shows the values in millions.) The effect of human players on the trading volume is less obvious. In our experiments, human participants sometimes increased the volume with respect to the identical run; sometimes the value stayed the same, while in other cases it was actually lowered. However, these last cases corresponded with the huge initial market bubble discussed next, so perhaps they could be ascribed to computational agents adapting to an extreme initial market.

19 An early Agent-Based stock market: replication and participation 65 Despite the increased level of market deviations, in the long run price always oscillated around fundamental value. This suggests that the artificial agents are able to adapt to the behavior of humans, but it is also clear that, quite naturally, their ability to keep the system close to equilibrium is limited. This latter statement is even clearer in the light of Fig. 8 summarizing the very first run of our experiments (not included in Fig. 7). What happened here was that inexperienced human participants wanted to buy unanimously, so the price skyrocketed. Their quest for shares is understandable psychologically, considering that they had money (200 units) and the price was low (around 80 unit). The figure also shows that computational agents were able to bring the system back towards equilibrium. However, a detailed look at the logs reveal that this happened only after all the human buyers backed down. This happened despite that humans only represented a mere 8% of the traders. The results of this very first run also provided a good example of the effects of learning by humans. In the subsequent runs, the initial price-peak gradually diminished, as demonstrated by the graphs of Fig. 9. In fact, most players reflected on this episode in their questionnaires, mentioning that they deliberately avoided buying in the beginning periods of later runs. a) 8 players of 100 agents b) 100 computational agents Fig. 8. Price (smooth line) and fundamental value (crossed line) history of the first experimental run, and that of the corresponding simulation The effect of human learning is also obvious in the performance (relative wealth) of the agents. Before studying these findings in more detail, let s consider a few important issues. As discussed in Subsection 4.1, the wealth of the agents grows rapidly. Moreover, due to the combination of the limited supply of shares and frequent payment of interests and dividends, the importance of stock market profit diminishes relative to the gross wealth. Because of this, the performance is path dependent in the sense that early successes overweight later ones. This

20 66 László Gulyás, Adamcsek Balázs and Árpád Kiss a) Second experimental run: b) Computational run corresponding 8 players of 100 agents to the second experimental run c) The third experimental run: d) Computational run corresponding 8 players of 20 agents to the third experimental run Fig. 9. The diminishing peak in the runs subsequent to that shown on Fig. 8

21 An early Agent-Based stock market: replication and participation 67 makes performance comparisons difficult, as the longer reaction time of human players further shortens the period when decisive profits can be made. On the other hand, computational agents always start from a random set of rules, and thus require time to evolve robust strategies. This factor may or may not be enough to balance the previous disadvantage of humans. a) The first experimental run, also shown on Fig. 8. (8 players of 100 agents) b) The second experimental run. (8 players of 100 agents) Fig. 10. The performance of human and computational agents in the first two experimental runs Despite all these concerns, Fig. 10 and Fig. 11 show that computational agents outperformed humans in the initial runs, but later the players improved significantly. The graphs display the normalized wealth of the worst and the best computational and human agents, together with the average normalized wealth of each type. By normalized wealth we mean the ratio of the agents wealth

22 68 László Gulyás, Adamcsek Balázs and Árpád Kiss and the base wealth (defined in Subection 4.1). Note that as agents can be indebted, this measure can be negative, too. Fig. 10 a) shows the result of the large market deviation of the first run. Clearly, even the best human participant performed worse than the worst computational agent. This situation improved in the second run (see graph b)) with the average human wealth being about equal to the wealth of the worst performing artificial agent. This demonstrates the effect of human agents carrying their knowledge over to the next run. In the two later runs shown on Fig. 11, the best performing agent was a human. On graph a) the average human wealth was around the maximum computational performance. On graph b), however, average human performance is only slightly better than that of the artificial agents. This may be due to the different number of participating agents in the two runs. There are a number of striking regularities on the graphs of Fig. 10 and Fig. 11. First, the average of computational performances always appears to be very close to unity. If there weren t human players present, this would be a natural consequence of the closed system. With human participants, however, this points to a potentially interesting phenomenon. Moreover, it was a human participant, in all four cases, who gave the worst performance. This suggests that while some participants learned quickly, some others failed to develop a viable strategy. Finally, the wealth of worst and best human players appears to follow a similar path on the two graphs of Fig. 10. Most likely, this is the side effect of the market bubbles shown on Fig. 8 a) and on Fig. 9 a). Human players were caught up in buying during the initial period, and they paid a serious price when the bubble burst. The ultimately most interesting findings, however, may lie beneath the surface of the results above. It is connected to the strategies developed by the participants. According to [1], theorists and market traders have strikingly different views about markets. Standard theory assumes identical investors who share their rational expectations about an asset s future price. Consequently, speculation cannot be profitable, except by luck; trading volume stays low, and market bubbles and crashes reflect rational changes in the asset s valuation. In contrast, it is needless to argue that traders do speculate in practice. Also, market deviations are often ascribed to market psychology. There is also an interpretation of these differences at the level of practical rules. If speculation works, technical rules that are based on only price or trade volume information may be useful. According to rational expectations theory, however, only fundamental strategies that relate price to fundamental value by using dividend information will yield success. Based on their answers to the questionnaire, initially all participants applied technical trading rules. They bought if the price was low, and sold if it was high. Then, gradually, a few of them discovered fundamental strategies. Finally, the winning strategy of Fig. 11 was perhaps the purest of fundamental strategies: buy if price < fundamental value, sell if it is the other way around. These early results suggest that technical trading lends itself easily to inexperienced traders, but fundamental strategies perform better in this artificial stock market.