Stock Market Trading via Stochastic Network Optimization

Transcription

1 PROC. IEEE CONFERENCE ON DECISION AND CONTROL (CDC), ATLANTA, GA, DEC Stock Market Tradig via Stochastic Network Optimizatio Michael J. Neely Uiversity of Souther Califoria mjeely Abstract We cosider the problem of dyamic buyig ad sellig of shares from a collectio of N stocks with radom price fluctuatios. To limit ivestmet risk, we place a upper boud o the total umber of shares kept at ay time. Assumig that prices evolve accordig to a ergodic process with a mild decayig memory property, ad assumig costraits o the total umber of shares that ca be bought ad sold at ay time, we develop a tradig policy that comes arbitrarily close to achievig the profit of a ideal policy that has perfect kowledge of future evets. Proximity to the optimal profit comes with a correspodig tradeoff i the maximum required stock level ad i the timescales associated with covergece. We the cosider arbitrary (possibly o-ergodic) price processes, ad show that the same algorithm comes close to the profit of a frame based policy that ca look a fixed umber of slots ito the future. Our approach uses a Lyapuov optimizatio techique previously developed for optimizig stochastic queueig etworks. Idex Terms Queueig aalysis, stochastic cotrol, uiversal algorithms I. INTRODUCTION This paper cosiders the problem of stock tradig i a ecoomic market with N stocks. We treat the problem i discrete time with ormalized time slots t {0, 1, 2,...}, where buyig ad sellig trasactios are coducted o each slot. Let Q(t) = (Q 1 (t),..., Q N (t)) be a vector of the curret umber of shares owed of each stock, called the stock queue. That is, for each {1,..., N}, the value of Q (t) is a iteger that represets the umber of shares of stock. Stock prices are give by a vector p(t) = (p 1 (t),..., p N (t)) ad are assumed to evolve radomly, with mild assumptios to be made precise i later sectios. Each buy ad sell trasactio icurs tradig costs. Stocks ca be sold ad purchased o every slot. Let φ(t) represet the et profit o slot t (after trasactio costs are paid). The goal is to desig a tradig policy that maximizes the log term time average of φ(t). For this system model, we eforce the additioal costrait that at most shares of each stock ca be bought ad sold o a give slot. This esures that our tradig decisios oly gradually chage the portfolio allocatio. While this costrait ca sigificatly limit the ability to take advatage of desirable prices, ad hece limits the maximum possible log term profit, we show that it ca also reduce ivestmet risk. Specifically, subject to the costrait, we develop a algorithm that achieves a time average profit This material is supported i part by oe or more of the followig: the DARPA IT-MANET program grat W911NF , the NSF Career grat CCF that is arbitrarily close to optimal, with a tradeoff i the maximum umber of shares Q max required for stock. The Q max values ca be chose as desired to limit the losses from a potetial collapse of oe or more of the stocks. It also impacts the timescales over which profit is accumulated, where smaller Q max levels lead to faster covergece times. It is importat to ote that log term wealth typically grows expoetially whe the Q max ad costraits are removed. I cotrast, it ca be show that these Q max ad costraits restrict wealth to at most a liear growth. Therefore, usig Q max ad to limit ivestmet risk ufortuately has a dramatic impact o the log term growth curve. However, our ability to boud the timescales over which wealth is eared suggests that our strategy may be useful i cases whe, i additio to a good log-term retur, we also desire oticeable ad cosistet short-term gais. At the ed of this paper, we briefly describe a modified strategy that icreases Q max ad as wealth progresses, with the goal of achievig oticeable short-term gais while eablig expoetial wealth icrease. Our approach uses the Lyapuov optimizatio theory developed for stochastic queueig etworks i our previous work [2][3][4]. Specifically, the work [2][3][4] develops resource allocatio ad schedulig policies for commuicatio ad queueig etworks with radom traffic ad chaels. The policies ca maximize time average throughput-utility ad miimize time average power expediture, as well as optimize more geeral time average attributes, without a-priori kowledge of the traffic ad chael probabilities. The algorithms cotiuously adapt to emergig coditios, ad are robust to o-ergodic chages i the probability distributios [5]. This suggests that similar cotrol techiques ca be used successfully for stock tradig problems. The differece is that the queues associated with stock shares are cotrolled to have positive drift (pushig them towards the maximum queue size), rather tha egative drift (which would push them i the directio of the empty state). The Dyamic Tradig Algorithm that we develop from these techiques ca be ituitively viewed as a variatio o a theme of dollar cost averagig, where price dowturs are exploited by purchasig more stock. However, the actual amout of stock that we buy ad sell o each slot is determied by a costraied optimizatio of a max-weight fuctioal that icorporates trasactio costs, curret prices, ad curret stock queue levels. Much prior work o fiacial aalysis ad portfolio opti-

2 PROC. IEEE CONFERENCE ON DECISION AND CONTROL (CDC), ATLANTA, GA, DEC mizatio assumes a kow probability model for stock prices. Classical portfolio optimizatio techiques by Markowitz [6] ad Sharpe [7] costruct portfolio allocatios over N stocks to maximize profit subject to variace costraits (which model risk) over oe ivestmet period (see also [8] ad refereces therei). Solutios to this problem ca be calculated if the mea ad covariace of stock returs are kow. Samuelso cosiders multi-period problems i [9] usig dyamic programmig, assumig a kow product form distributio for ivestmet returs. Cover i [10] develops a iterative procedure that coverges to the costat portfolio allocatio that maximizes the expected log ivestmet retur, assumig a kow probability distributio that is the same o each period. Recet work by Rudoy ad Rohrs i [11] [12] cosiders riskaware optimizatio with a more complex coitegrated vector autoregressive assumptio o stock processes, ad uses Mote Carlo simulatios over historical stock trajectories to iform stochastic decisios. Stochastic models of stock prices usig Lévy processes ad multi-fractal processes are cosidered i [8] [13] [14] ad refereces therei. A sigificat departure from this work is the uiversal stock tradig paradigm, as exemplified i prior works of Cover ad Gluss [15], Larso [16], Cover [17], Merhav ad Feder [18], ad Cover ad Ordetlich [19] [20], where tradig algorithms are developed ad show to provide aalytical guaratees for ay sample path of stock prices. Specifically, the work i [15]-[20] seeks to fid a o-aticipatig tradig algorithm that yields the same growth expoet as the best costat portfolio allocatio, where the costat ca be optimized with full kowledge of the future. The works i [15][16] develop algorithms that come close to the optimal expoet, ad the work i [17] achieves the optimal expoet uder mild assumptios that boud price sample paths away from zero. Similar results are derived i [18] usig a geeral framework of sequetial decisio theory. Related results are derived i [19] [20], where [19] treats problems with additioal side iformatio, ad [20] treats max-mi performace whe stock prices are chose by a adversary. Our work is similar i spirit to this uiversal tradig paradigm, i that we do ot base decisios o a kow (or estimated) probability distributio. However, our cotext, solutio methodology, ad solutio structure is very differet. Ideed, the works i [15]-[20] assume that the etire stock portfolio ca be sold ad reallocated o every time period, ad allow stock holdigs to grow arbitrarily large. This meas that the accumulated profit is always at risk of oe or more stock failures. I our work, we take a more coservative approach that restricts reallocatio to gradual chages, ad that pockets profits while holdig o more tha Q max shares of each stock. We also explicitly accout for tradig costs ad iteger costraits o stock shares, which is ot cosidered i the works [15]-[20]. I this cotext, we first desig a algorithm uder the assumptio that prices are ergodic with a ukow distributio. I this case, we develop a simple o-aticipatig algorithm that comes arbitrarily close to the optimal time average profit that could be eared by a ideal policy with complete kowledge of the future. The ideal policy used for compariso ca make differet allocatios at differet times, ad is ot restricted to costat allocatios. We the show that the same algorithm ca be used for geeral price sample paths, eve o-ergodic sample paths without well defied time averages. A more coservative guaratee is show i this case: The algorithm yields profit that is arbitrarily close to that of a frame based policy with T - slot lookahead, where the future is kow up to T slots. Our approach is differet from prior work o uiversal algorithms ad is ispired by Lyapuov optimizatio ad decisio theory for stochastic queueig etworks [2]. However, the Lyapuov theory we use here ivolves sample path techiques that are differet from those i [2]. These techiques might have broader impacts o queueig problems i other areas. I the ext sectio we preset the system model. I Sectio III we develop the Dyamic Tradig Algorithm ad aalyze performace for the simple (ad possibly urealistic) case whe price vectors p(t) are ergodic ad i.i.d. over slots. While this i.i.d. case does ot accurately model actual stock prices, its aalysis provides valuable isight. Sectio IV shows the algorithm also provides performace guaratees for completely arbitrary price processes (possibly o-ergodic). Simple extesios are described i Sectio V. II. SYSTEM MODEL Let A(t) = (A 1 (t),..., A N (t)) be a vector of decisio variables represetig the umber of ew shares purchased for each stock o slot t, ad let µ(t) = (µ 1 (t),..., µ N (t)) be a vector represetig the umber of shares sold o slot t. The values A (t) ad µ (t) are o-egative itegers for each {1,..., N}. Each purchase of A ew shares of stock icurs a trasactio cost b (A) (called the buyig cost fuctio). Likewise, each sale of µ shares of stock icurs a trasactio cost s (µ) (called the sellig cost fuctio). The fuctios b (A) ad s (µ) are arbitrary, ad are assumed oly to satisfy b (0) = s (0) = 0, ad to be o-egative, odecreasig, ad bouded by fiite costats b max ad s max, so that: 0 b (A) b max 0 s (µ) s max for 0 A for 0 µ where for each {1,..., N}, is a positive iteger that limits the amout of shares of stock that ca be bought ad sold o slot t. A. Example Trasactio Cost Fuctios The fuctios b (A) might be liear, represetig a trasactio fee that charges per share purchased. Aother example is a fixed cost model with some fixed positive fee b, so that: { b if A > 0 b (A) = 0 if A = 0 Similar models ca be used for the s (µ) fuctio. The simplest model of all is the zero trasactio cost model where the fuctios b (A) ad s (µ) are idetically zero.

3 PROC. IEEE CONFERENCE ON DECISION AND CONTROL (CDC), ATLANTA, GA, DEC B. System Dyamics The stock price vector p(t) is assumed to be a radom vector process that takes values i some fiite set P R N, where P ca have a arbitrarily large umber of elemets. 1 For each, let p max represet a boud o p (t), so that: 0 p (t) p max for all t ad all {1,..., N} (1) We assume that buyig ad sellig decisios ca be made o each slot t based o kowledge of p(t). The sellig decisio variables µ(t) are made every slot t subject to the followig costraits: µ (t) {0, 1,..., } for all {1,..., N} (2) µ (t)p (t) s (µ (t)) for all {1,..., N} (3) µ (t) Q (t) for all {1,..., N} (4) Costrait (2) esures that o more tha shares ca be sold of stock o a sigle slot. Costrait (3) restricts to the reasoable case whe the moey eared from the sale of a stock must be larger tha the trasactio fee associated with the sale (violatig this costrait would clearly be suboptimal). 2 Costrait (4) requires the umber of shares sold to be less tha or equal to the curret umber owed. The buyig decisio variables A(t) are costraied as follows: A (t) {0, 1, 2,..., } for all {1,..., N} (5) =1 A (t)p (t) x (6) where x is a positive value that bouds the total amout of moey used for purchases o slot t. For simplicity, we assume there is always at least a miimum of x ad =1 [µmax )] dollars available for makig purchasig decisios. This model ca be augmeted by addig a checkig accout queue Q 0 (t) from which we must draw moey to make purchases, although we omit this aspect for brevity. The resultig queueig dyamics for the stock queues Q (t) for {1,..., N} are thus: p max + b ( Q (t + 1) = max[q (t) µ (t) + A (t), 0] (7) Strictly speakig, the max[, 0] operator i the above dyamic equatio is redudat, because the costrait (4) esures that the argumet iside the max[, 0] operator is o-egative. However, the max[, 0] shall be useful for mathematical aalysis whe we compare our strategy to that of a queueidepedet strategy that eglects costrait (4). 1 The cardiality of the set P does ot eter ito our aalysis. We assume it is fiite oly for the coveiece of claimig that the supremum time average profit φ opt is achievable by a sigle p-oly policy, as described i Sectio II-D. Theorems 1, 2 are uchaged if the set P is ifiite, although the proofs of Theorem 1 would require a additioal limitig argumet over p- oly policies that approach φ opt. 2 Costrait (3) ca be augmeted by allowig equality oly if µ (t) = 0. C. The Maximum Profit Objective Defie φ(t) as the et profit o slot t: φ(t) = [µ (t)p (t) s (µ (t))] =1 [A (t)p (t) + b (A (t))] (8) =1 Defie φ as the time average expected value of φ(t) uder a give tradig algorithm (temporarily assumed to have a well defied limit): φ= 1 t 1 lim E {φ(τ)} t t The goal is to desig a tradig policy that maximizes φ. It is clear that the trivial strategy that chooses µ(t) = A(t) = 0 for all t yields φ(t) = 0 for all t, ad results i φ = 0. Therefore, we desire our algorithm to produce a log term profit that satisfies φ > 0. D. The Stochastic Price Vector ad p-oly Policies We first assume the stochastic process p(t) has well defied time averages (this is geeralized to o-ergodic models i Sectio IV). Specifically, for each price vector p i the fiite set P, we defie π(p) as the time average fractio of time that p(t) = p, so that: 1 t 1 lim 1{p(τ) = p} = π(p) with probability 1 (9) t t where 1{p(τ) = p} is a idicator fuctio that is 1 if p(τ) = p, ad zero otherwise. Defie a p-oly policy as a buyig ad sellig strategy that chooses virtual decisio vectors A (t) ad µ (t) as a statioary ad possibly radomized fuctio of p(t), costraied oly by (2)-(3) ad (5)-(6). That is, the virtual decisio vectors A (t) ad µ (t) associated with a p-oly policy do ot ecessarily satisfy the costrait (4) that is required of the actual decisio vectors, ad hece these decisios ca be made idepedetly of the curret stock queue levels. Uder a give p-oly policy, defie the followig time average expectatios d ad φ : d = 1 t 1 lim E {A t t (τ) µ (τ)} (10) { φ = 1 t 1 lim E [µ t t (τ)p (τ) s (µ (τ))] =1 } [A (τ)p (τ) + b (A (τ))] =1 (11) It is easy to see by (9) that these time averages are well defied for ay p-oly policy. For each, the value d represets the virtual drift of stock queue Q (t) associated with the virtual decisios A (t) ad µ (t). The value φ represets the virtual profit uder virtual decisios A (t) ad µ (t). Note that the

4 PROC. IEEE CONFERENCE ON DECISION AND CONTROL (CDC), ATLANTA, GA, DEC trivial p-oly policy A (t) = µ (t) = 0 yields d = 0 for all, ad φ = 0. Thus, we ca defie φ opt as the supremum value of φ over all p-oly policies that yield d 0 for all, ad we ote that φ opt 0. Usig a argumet similar to that give i [3], it ca be show that: 1) φ opt is achievable by a sigle p-oly policy that satisfies d = 0 for all {1,..., N}. 2) φ opt is greater tha or equal to the supremum of the lim sup time average expectatio of φ(t) that ca be achieved over the class of all actual policies that satisfy the costraits (2)-(6), icludig ideal policies that use perfect iformatio about the future. Thus, o policy ca do better tha φ opt. That φ opt is achievable by a sigle p-oly policy (rather tha by a limit of a ifiite sequece of policies) ca be show usig the assumptio that the set P of all price vectors is fiite. That φ opt bouds the time average profit of all policies, icludig those that have perfect kowledge of the future, ca be ituitively uderstood by otig that the optimal profit is determied oly by the time averages π(p). These time averages are the same (with probability 1) regardless of whether or ot we kow the future. The detailed proofs of these results are similar to those i [3] ad are provided i [1]. I the ext sectio we develop a Dyamic Tradig Algorithm that satisfies the costraits (2)-(6) ad that does ot kow the future or the distributio π(p), yet yields time average profit that is arbitrarily close to φ opt. To develop our Dyamic Tradig Algorithm, we first focus o the simple case whe the vector p(t) is idepedet ad idetically distributed (i.i.d.) over slots, with a geeral probability distributio π(p). This is a overly simplified model ad does ot reflect actual stock time series data. Ideed, a more accurate model would be to assume the differeces i the logarithm of prices are i.i.d. (see [8] ad refereces therei). However, we show i Sectio IV that the same algorithm is also efficiet for arbitrary (possibly o-ergodic) price models. E. The i.i.d. Model Suppose p(t) is i.i.d. over slots with P r[p(t) = p] = π(p) for all p P. Because the value φ opt is achievable by a sigle p-oly policy, ad because the expected values of ay p-oly policy are the same every slot uder the i.i.d. model, we have the followig: There is a p-oly policy A (t), µ (t) that yields for all t ad all Q(t): ad E {A (t) µ (t) Q(t)} = 0 (12) { N E =1 [µ (t)p (t) s (µ (t))] =1 [A (t)p (t) + b (A (t))] Q(t) } = φ opt (13) III. CONSTRUCTING A DYNAMIC TRADING ALGORITHM The goal is to esure that all stock queues Q (t) are maitaied at reasoably high levels so that there are typically eough shares available to sell if a opportue price should arise. To this ed, defie θ 1,..., θ as positive real umbers that represet target queue sizes for the stock queues (soo to be related to the maximum queue size). The particular values θ 1,..., θ shall be chose later. As a scalar measure of the distace each queue is away from its target value, we defie the followig Lyapuov fuctio L(Q(t)): L(Q(t)) = 1 2 (Q (t) θ ) 2 (14) =1 Suppose that Q(t) evolves accordig to some probability law, ad defie (Q(t)) as the oe-slot coditioal Lyapuov drift: 3 (Q(t)) =E {L(Q(t + 1)) L(Q(t)) Q(t)} (15) As i the stochastic etwork optimizatio problems of [2][3][4], our approach is to take cotrol actios o each slot t to miimize a boud o the drift-plus-pealty expressio: (Q(t)) V E {φ(t) Q(t)} where V is a positive parameter to be chose as desired to affect the proximity to the optimal time average profit φ opt. To this ed, we first compute a boud o the Lyapuov drift. Lemma 1: (Lyapuov drift boud) For all t ad all possible values of Q(t), we have: (Q(t)) B (Q (t) θ )E {µ (t) A (t) Q(t)} =1 where B is a fiite costat that satisfies: B 1 2 E { (µ (t) A (t)) 2 Q(t) } (16) =1 Such a fiite costat B exists because of the boudedess assumptios o buy ad sell variables µ (t) ad A (t). I particular, we have: B 1 ( ) 2 =D (17) 2 =1 Proof: The proof follows by squarig the queue dyamics (7) (see [1]). Usig Lemma 1 with the defiitio of φ(t) i (8), a boud o the drift-mius-reward expressio is give as follows: (Q(t)) V E {φ(t) Q(t)} B =1 (Q (t) θ )E {µ (t) A (t) Q(t)} V =1 E {µ (t)p (t) s (µ (t)) Q(t)} +V =1 E {A (t)p (t) + b (A (t)) Q(t)} (18) We desire a algorithm that, every slot, observes the Q(t) values ad the curret prices, ad makes a greedy tradig actio subject to the costraits (2)-(6) that miimizes the right-had-side of (18). 3 Strictly speakig, proper otatio is (Q(t), t), as the drift may arise from a o-statioary algorithm. However, we use the simpler otatio (Q(t)) as a formal represetatio of the right-had-side of (15).

5 PROC. IEEE CONFERENCE ON DECISION AND CONTROL (CDC), ATLANTA, GA, DEC A. The Dyamic Tradig Algorithm Every slot t, observe Q(t) ad p(t) ad perform the followig actios. 1) Sellig: For each {1,..., N}, choose µ (t) to solve: Miimize: [θ Q (t) V p (t)]µ (t) + V s (µ (t)) Subject to: Costraits (2)-(4) 2) Buyig: Choose A(t) = (A 1 (t),..., A (t)) to solve: Miimize: =1 [Q (t) θ + V p (t)]a (t) + =1 V b (A (t)) Subject to: Costraits (5)-(6) The buyig algorithm uses the iteger costraits (5)-(6), ad is related to the well kow bouded kapsack problem (it is exactly the bouded kapsack problem if the b ( ) fuctios are liear). Implemetatio of this iteger costraied problem ca be complex whe the umber of stocks N is large. However, if we use x= =1 [µmax p max + b ( )], the costrait (6) is effectively removed. I this case, the stocks are decoupled ad the buyig algorithm reduces to makig separate decisios for each stock. Alteratively, the costrait (6) ca be replaced by the costrait =1 A (t) A tot, where A tot is a iteger that bouds the total umber of stocks that ca be bought o a sigle slot. I this case, it is easy to see that if buyig costs are liear, so that b (A) = b A for all (for some positive costats b ), the the buyig algorithm reduces to successively buyig as much stock as possible from the queues with the smallest (ad egative) [Q (t) θ +V (p (t)+b )] values. A alterative relaxatio of the costrait (6) is discussed i [1]. Lemma 2: For a give Q(t) o slot t, the above dyamic tradig algorithm satisfies: B V φ(t) (Q (t) θ )(µ (t) A (t)) =1 B V φ (t) (Q (t) θ )(µ (t) A (t)) (19) =1 where A(t), µ(t) are the actual decisios made by the algorithm, which defie φ(t) by (8), ad A (t), µ (t) are ay alterative (possibly radomized) decisios that ca be made o slot t that satisfy (2)-(6), which defie φ (t) by (8). Furthermore, we have: (Q(t)) V E {φ(t) Q(t)} B =1 (Q (t) θ )E {µ (t) A (t) Q(t)} V =1 E {µ (t)p (t) s (µ (t)) Q(t)} +V =1 E {A (t)p (t) + b (A (t)) Q(t)} (20) where the expectatio o the right-had-side of (20) is with respect to the radom price vector p(t) ad the possibly radom actios A (t), µ (t) i respose to this price vector. Proof: Give Q(t) o slot t, the dyamic algorithm makes buyig ad sellig decisios to miimize the left-had-side of (19) over all alterative decisios that satisfy (2)-(6). Therefore, the iequality (19) holds for all realizatios of the radom quatities, ad hece also holds whe takig coditioal expectatios of both sides. The coditioal expectatio of the left-had-side of (19) is equivalet to the right-had-side of the drift-mius-reward expressio (18), which proves (20). The mai idea behid our aalysis is that the Dyamic Tradig Algorithm is simple to implemet ad does ot require kowledge of the future or of the statistics of the price process p(t). However, it ca be compared to alterative policies A (t) ad µ (t) (such as i Lemma 2), ad these policies possibly have kowledge both of the price statistics ad of the future. B. Boudig the Stock Queues The ext lemma shows that the above algorithm does ot sell ay shares of stock if Q (t) is sufficietly small. Lemma 3: Uder the above Dyamic Tradig Algorithm ad for arbitrary price processes p(t) that satisfy (1), if Q (t) < θ V p max for some particular queue ad slot t, the µ (t) = 0. Therefore, if Q (0) θ V p max, the: Q (t) θ V p max for all t Proof: Suppose that Q (t) < θ V p max for some particular queue ad slot t. The for ay µ 0 we have: [θ Q (t) V p (t)]µ + V s (µ) [θ Q (t) V p max ]µ + V s (µ) [θ Q (t) V p max ]µ 0 where the fial iequality holds with equality if ad oly if µ = 0. Therefore, the Dyamic Tradig Algorithm must choose µ (t) = 0. Now suppose that Q (t) θ V p max for some time t. We show it also holds for t+1. If Q (t) θ V p max the it ca decrease by at most, o a sigle slot, so that Q (t+1) θ V p max. Coversely, if θ V p max > Q (t) θ V p max, the we kow µ (t) = 0 ad so the queue caot decrease o the ext slot ad we agai have Q (t + 1) θ V p max. It follows that this iequality is always upheld if it is satisfied at t = 0. We ote that the above lemma is a sample path statemet that holds for arbitrary (possibly o-ergodic) price processes. The ext lemma also deals with sample paths, ad shows that all queues have a fiite maximum size Q max. Lemma 4: Uder the above Dyamic Tradig Algorithm ad for arbitrary price processes p(t) that satisfy (1), if Q (t) > θ for some particular queue ad slot t, the A (t) = 0 ad so the queue caot icrease o the ext slot. It follows that if Q (0) θ +, the: Q (t) θ + for all t Proof: Suppose that Q (t) > θ for a particular queue ad slot t. Let A(t) = (A 1 (t),..., A N (t)) be a vector of buyig decisios that solve the optimizatio associated with the Buyig algorithm o slot t, so that they miimize the expressio: [Q m (t) θ m + V p m (t)]a m (t) + V b m (A m (t)) (21) m=1 m=1

6 PROC. IEEE CONFERENCE ON DECISION AND CONTROL (CDC), ATLANTA, GA, DEC subject to (5)-(6). Suppose that A (t) > 0 (we shall reach a cotradictio). Because the term [Q (t) θ + V p (t)] is strictly positive, ad because the b (A) fuctio is odecreasig, we ca strictly reduce the value of the expressio (21) by chagig A (t) to 0. This chage still satisfies the costraits (5)-(6) ad produces a strictly smaller sum i (21), cotradictig the assumptio that A(t) is a miimizer. Thus, if Q (t) > θ, the A (t) = 0. Because the queue value ca icrease by at most o ay slot, ad caot icrease if it already exceeds θ, it follows that Q (t) θ + for all t, provided that this iequality holds at t = 0. C. Aalyzig Time Average Profit Theorem 1: Fix ay value V > 0, ad defie θ as follows: θ = V p max Suppose that iitial stock queues satisfy: Q (0) V p max + 2 (22) + 3 (23) If the Dyamic Tradig Algorithm is implemeted over t {0, 1, 2,...}, the: (a) Stock queues Q (t) (for {1,..., N}) are determiistically bouded for all slots t as follows: Q (t) V p max + 3 for all ad all t (24) (b) If p(t) is i.i.d. over slots with geeral distributio P r[p(t) = p] = π(p) for all p P, the for all t {1, 2,...} we have: φ(t) φ opt B V E {L(Q(0))} (25) V t where the costat B is defied by (16) (ad satisfies the iequality (17)), φ opt is the optimal time average profit, ad φ(t) is the time average expected profit over t slots: φ(t) = 1 t 1 t E {φ(τ)} (26) Therefore: lim if φ(t) t φopt B/V (27) Theorem 1 shows that the time average expected profit is withi B/V of the optimal value φ opt. Because the B costat is idepedet of V, we ca choose V to make B/V arbitrarily small. This comes with a tradeoff i the maximum size required for each stock queue that is liear i V. Specifically, the maximum stock level Q max required for stock is give as follows: Q max =V p max + 3 Now suppose that we start with iitial coditio Q (0) = for all ad all t. The for t {1, 2,...} the error term L(Q(0))/(V t) is give by: =1 (V pmax L(Q(0)) + ) 2 = = O(V )/t (28) V t 2V t This shows that if V is chose to be large, the the amout of time t required to make this error term egligible must also be large. Oe ca miimize this error term with a iitial coditio Q (0) that is close to θ for all. However, this is a artificial savigs, as it does ot iclude the startup cost associated with purchasig that may iitial uits of stock. Therefore, the timescales are more accurately described by the trasiet give i (28). Oe may woder how the Dyamic Tradig Algorithm is achievig ear optimal profit without kowig the distributio of the price vector p(t), ad without estimatig this distributio. The aswer is that it uses the queue values themselves to guide decisios. These queue values Q (t) oly deviate sigificatly from the target θ whe iefficiet decisios are made. The values the act as a sufficiet statistic o which to base future decisios. The same sufficiet statistic holds for the o-i.i.d. case, as show i Sectio IV, so that we do ot eed to estimate price patters or time-correlatios, provided that we allow for a sufficietly large cotrol parameter V ad correspodig large timescales for covergece. Fially, oe may also woder if the limitig time average expected profit give i (27) also holds (with probability 1) for the limitig time average profit (without the expectatio). Whe p(t) evolves accordig to a fiite state irreducible Markov chai (as is the case i this i.i.d. sceario), the the Dyamic Tradig Algorithm i tur makes Q(t) evolve accordig to a fiite state Markov chai, ad it ca be show that the limitig time average expected profit is the same (with probability 1) as the limitig time average profit. D. Proof of Theorem 1 Proof: (Theorem 1 part (a)) By Lemma 3 we kow that Q (t) θ V p max for all t (provided that this holds at t = 0). However, θ V p max =. Thus, Q (t) for all t, provided that this holds for t = 0. Similarly, by Lemma 4 we kow that Q (t) θ + for all t (provided that this holds for t = 0), ad θ + Q max. = Proof: (Theorem 1 part (b)) Fix a slot t {0, 1, 2,...}. To prove part (b), we plug a alterative set of cotrol choices A (t) ad µ (t) ito the drift-mius-reward boud (20) of Lemma 2. Because p(t) is i.i.d., we ca choose A (t) ad µ (t) as the p-oly policy that satisfies (12), (13). Note that we must first esure this p-oly policy satisfies the costrait (4) eeded to apply the boud (20). However, we kow from part (a) of this theorem that Q (t) for all, ad so the costrait (4) is trivially satisfied. Therefore, we ca plug this policy A (t) ad µ (t) ito (20) ad use equalities (12) ad (13) to yield: (Q(t)) V E {φ(t) Q(t)} B V φ opt Takig expectatios of the above iequality over the distributio of Q(t) ad usig the law of iterated expectatios yields: E {L(Q(t + 1)) L(Q(t))} V E {φ(t)} B V φ opt The above holds for all t {0, 1, 2,..., }. Summig the above over τ {0,..., t 1} (for some positive iteger t) yields: t 1 E {L(Q(t)) L(Q(0))} V E {φ(τ)} tb tv φ opt

7 PROC. IEEE CONFERENCE ON DECISION AND CONTROL (CDC), ATLANTA, GA, DEC Dividig by tv, rearragig terms, ad usig o-egativity of L( ) yields: φ(t) φ opt B/V E {L(Q(0))} /V t where φ(t) is defied i (26). This proves the result. IV. ARBITRARY PRICE PROCESSES The Dyamic Tradig Algorithm ca be show to provide similar performace for more geeral o-i.i.d. processes that are ergodic with a mild decayig memory property [1]. Here we cosider the performace of the Dyamic Tradig Algorithm for a arbitrary price vector process p(t), possibly a o-ergodic process without a well defied time average such as that give i (9). I this case, there may ot be a well defied optimal time average profit φ opt. However, oe ca defie φ opt (t) as the maximum possible time average profit achievable over the iterval {0,..., t 1} by a algorithm with perfect kowledge of the future ad that coforms to the costraits (2)-(6). For the ergodic settigs described i the previous sectios, φ opt (t) has a well defied limitig value, ad our algorithm comes close to its limitig value. I this (possibly o-ergodic) settig, we do ot claim that our algorithm comes close to φ opt (t). Rather, we make a less ambitious claim that our policy yields a profit that is close to (or greater tha) the profit achievable by a frame-based policy that ca look oly T slots ito the future. A. The T -Slot Lookahead Performace Let T be a positive iteger, ad fix ay slot t 0 {0, 1, 2,...}. Defie ψ T (t 0 ) as the optimal profit achievable over the iterval {t 0,..., t 0 + T 1} by a policy that has perfect a-priori kowledge of the prices p(τ) over this iterval, ad that esures for each {1,..., N} that the total amout of stock purchased over this iterval is greater tha or equal to the total amout sold. Specifically, ψ T (t 0 ) is mathematically defied accordig to the followig optimizatio problem that has decisio variables A(τ), µ(τ), ad that treats the stock prices p(τ) as determiistically kow quatities: Max: Subj. to: =1 [µ (τ)p (τ) s (µ (τ))] ψ= t 0+T 1 t 0+T 1 =1 [A (τ)p (τ) + b (A (τ))] (29) t0+t 1 A (τ) = t 0+T 1 µ (τ) (30) Costraits (2), (3), (5), (6) (31) The value ψ T (t 0 ) is equal to the maximizig value ψ i the above problem (29)-(31). Note that the costrait (30) oly requires the amout of type- stock purchased to be equal to the amout sold by the ed of the T -slot iterval, ad does ot require this at itermediate steps of the iterval. This allows the T -slot lookahead policy to sell short stock that is ot yet owed, provided that the requisite amout is purchased by the ed of the iterval. Note that the trivial decisios A(τ) = µ(τ) = 0 for τ {t 0,..., t 0 +T 1} lead to 0 profit over the iterval, ad hece Ψ T (t 0 ) 0 for all T ad all t 0. Cosider ow the iterval {0, 1,..., MT 1} that is divided ito a total of M frames of T -slots. We show that for ay positive iteger M, our Dyamic Tradig Algorithm yields a average profit over this iterval that is close to the average profit of a T -slot lookahead policy that is implemeted o each T -slot frame of this iterval. B. The T -Slot Sample Path Drift Let L(Q(t)) be the Lyapuov fuctio of (14). For a give slot t ad a give positive iteger T, defie the T -slot sample path drift ˆ T (t) as follows: ˆ T (t) =L(Q(t + T )) L(Q(t)) (32) This differs from the oe-slot coditioal Lyapuov drift i (15) i two respects: It cosiders the differece i the Lyapuov fuctio over T slots, rather tha a sigle slot. It is a radom variable equal to the differece betwee the Lyapuov fuctio o slots t ad t + T, rather tha a coditioal expectatio of this differece. Lemma 5: Suppose the Dyamic Tradig Algorithm is implemeted, with θ values satisfyig (22), ad iitial coditio that satisfies (23). The for ay give slot t 0 ad all itegers T > 0, we have: ˆ T (t 0 ) V t 0+T 1 φ(τ) DT 2 V t 0+T 1 φ (τ) =1 [Q (t 0 ) θ ] t 0+T 1 [µ (τ) A (τ)] where the costat D is defied i (17), φ(τ) is defied i (8), ad φ (τ), µ (τ), A (τ) represet ay alterative cotrol actios for slot τ that satisfy the costraits (2), (3), (5), (6). Proof: See [1][21] (the D costat stated here uses a techique i [21] to improve the boud give i [1]). Theorem 2: Suppose the Dyamic Tradig Algorithm is implemeted, with θ values satisfyig (22), ad iitial coditio that satisfies (23). The for ay arbitrary price process p(t) that satisfies (1), we have: (a) All queues Q (t) are bouded accordig to (24). (b) For ay positive itegers M ad T, the time average profit over the iterval {0,..., MT 1} satisfies the determiistic boud: MT 1 1 φ(τ) MT M 1 1 ψ T (mt ) MT m=0 DT V L(Q(0)) (33) MT V where the ψ T (mt ) values are defied accordig to the T - slot lookahead policy that uses kowledge of the future to solve (29)-(31) for each T -slot frame. The costat D is defied i (17), ad if Q(0) = ( 1,..., N ) the L(Q(0))/(MT V ) has the form (28) with t = MT. Proof: Part (a) has already bee prove i Theorem 1. To prove part (b), fix ay slot t 0 ad ay positive iteger T. Defie A (τ) ad µ (τ) as the solutio of (29)-(31) over the iterval τ {t 0,..., t 0 + T 1}. By (31), these decisio variables satisfy costraits (2), (3), (5), (6), ad hece ca be plugged i to the boud i Lemma 5. Because (29), (30) hold for these variables, by Lemma 5 we have: ˆ T (t 0 ) V t 0+T 1 φ(τ) DT 2 V ψ T (t 0 )

8 PROC. IEEE CONFERENCE ON DECISION AND CONTROL (CDC), ATLANTA, GA, DEC Usig the defiitio of ˆ T (t 0 ) give i (32) yields: L(Q(t 0 +T )) L(Q(t 0 )) V t 0+T 1 φ(τ) DT 2 V ψ T (t 0 ) The above iequality holds for all slots t 0 {0, 1, 2,...}. Lettig t 0 = mt ad summig over m {0, 1,..., M 1} (for some positive iteger M) yields: L(Q(MT )) L(Q(0)) V MDT 2 V MT 1 M 1 φ(τ) ψ T (mt ) m=0 Rearragig terms ad usig o-egativity of L( ) proves the theorem. Theorem 2 is stated for geeral price processes, but has explicit performace bouds for queue size i terms of the chose V parameter, ad for profit i terms of V ad of the profit ψ T (mt ) of T -slot lookahead policies. Pluggig a large value of T ito the boud (33) icreases the first term o the right-had-side because it allows for a larger amout of lookahead. However, this comes with the cost of icreasig the term DT/V that is required to be small to esure close proximity to the desired profit. Oe ca use this theorem with ay desired model of stock prices to compute statistics associated with ψ T (mt ) ad hece uderstad more precisely the timescales over which ear-optimal profit is achieved. V. EXTENSIONS Theorems 1 ad 2 require a iitial stock level of at least i all of the N stocks. This ca be achieved by iitially purchasig these shares (say, at time t = 1). It turs out that we ca achieve the same performace as specified i Theorems 1 ad 2 without payig this startup cost. This ca be doe usig the cocept of place-holder backlog from [22], which becomes place-holder stock i our cotext. This is detailed i [1]. Our techical report [1] also cosiders extesios to cases whe: (i) the maximum stock price observed jumps above our p max value, (ii) stock prices ca split, (iii) the θ ad Q max parameters are scaled as time progresses, so that expoetial wealth growth ca be achieved. VI. CONCLUSION This work uses Lyapuov optimizatio theory, developed for stochastic optimizatio of queueig etworks, to costruct a dyamic policy for buyig ad sellig stock. Whe prices are ergodic, a sigle o-aticipatig policy was costructed ad show to perform close to a ideal policy with perfect kowledge of the future, with a tradeoff i the required amout of stock kept i each queue ad i the timescales associated with covergece. For arbitrary price sample paths, the same algorithm was show to achieve a time average profit close to that of a frame based T -slot lookahead policy that ca look T slots ito the future. Our framework costrais the maximum umber of stock shares that ca be bought ad sold at ay time. While this restricts the log term growth curve to a liear growth, it also limits risk by esurig o more tha a costat value Q max shares of each stock are kept at ay time. A modified policy was briefly discussed that achieves expoetial growth by scalig Q max i proportio to icreased risk tolerace as wealth icreases. These results add to the theory of uiversal stock tradig, ad are importat for uderstadig optimal decisio makig i the presece of a complex ad possibly ukow price process. REFERENCES [1] M. J. Neely. Stock market tradig via stochastic etwork optimizatio. ArXiv Techical Report, arxiv: v1, Sept [2] L. Georgiadis, M. J. Neely, ad L. Tassiulas. Resource allocatio ad cross-layer cotrol i wireless etworks. Foudatios ad Treds i Networkig, vol. 1, o. 1, pp , [3] M. J. Neely. Eergy optimal cotrol for time varyig wireless etworks. IEEE Trasactios o Iformatio Theory, vol. 52, o. 7, pp , July [4] M. J. Neely. Dyamic Power Allocatio ad Routig for Satellite ad Wireless Networks with Time Varyig Chaels. PhD thesis, Massachusetts Istitute of Techology, LIDS, [5] M. J. Neely ad R. Urgaokar. Cross layer adaptive cotrol for wireless mesh etworks. Ad Hoc Networks (Elsevier), vol. 5, o. 6, pp , August [6] H. Markowitz. Portfolio selectio. Joural of Fiace, vol. 7, o. 1, pp , March [7] W. F. Sharpe. A simplified model for portfolio aalysis. Maagemet Sciece, vol. 9, o. 2, pp , Ja [8] J-P. Bouchaud ad M. Potters. Theory of Fiacial Risk ad Derivative Pricig: From Statistical Physics to Risk Maagemet, 2d ed. Cambridge Uiversity Press, [9] P. A. Samuelso. Lifetime portfolio selectio by dyamic stochastic programmig. The Review of Ecoomics ad Statistics, vol. 51, o. 3, pp , Aug [10] T. M. Cover. A algorithm for maximizig expected log ivestmet retur. IEEE Trasactios o Iformatio Theory, IT-30, pp , [11] M. B. Rudoy ad C. E. Rohrs. A dyamic programmig approach to two-stage mea-variace portfolio selectio i coitegrated vector autoregressive systems. IEEE Cof. o Decisio ad Cotrol, [12] M. B. Rudoy. Multistage Mea-Variace Portfolio Selectio i Coitegrated Vector Autoregressive Systems. PhD thesis, Massachusetts Istitute of Techology, Feb [13] A. Turiel ad C. J. Pérez-Vicete. Multifractal geometry i stock market time series. Physica A: Statistical Mechaics ad its Applicatios, vol. 322, pp , May [14] B. Madelbrot ad H. M. Taylor. O the distributio of stock price differeces. Operatios Research, vol. 15, o. 6, pp , [15] T. M. Cover ad D. Gluss. Empirical bayes stock market portfolios. Adv. Appl. Math, vol. 7, pp , [16] D. C. Larso. Growth Optimal Tradig Strategies. PhD thesis, Staford Uiversity, [17] T. M. Cover. Uiversal portfolios. Mathematical Fiace, vol. 1, o. 1, pp. 1-29, Ja [18] N. Merhav ad M. Feder. Uiversal schemes for sequetial decisio from idividual data sequeces. IEEE Trasactios o Iformatio Theory, vol. 39, o. 4, pp , July [19] T. M. Cover ad E. Ordetlich. Uiversal portfolios with side iformatio. IEEE Trasactios o Iformatio Theory, vol. 42, o. 2, [20] E. Ordetlich ad T. M. Cover. The cost of achievig the best portfolio i hidsight. Mathematics of Operatios Research, vol. 23, o. 4, Nov [21] M. J. Neely ad L. Huag. Dyamic product assembly ad ivetory cotrol for maximum profit. ArXiv Techical Report, arxiv: v1, April [22] M. J. Neely ad R. Urgaokar. Opportuism, backpressure, ad stochastic optimizatio with the wireless broadcast advatage. Asilomar Coferece o Sigals, Systems, ad Computers, Pacific Grove, CA, Oct