Sensitivity Analysis of Non-linear Performance with Probability Distortion

Transcription

1 Preprints of the 19th Worl Congress The International Feeration of Automatic Control Cape Town, South Africa. August 24-29, 214 Sensitivity Analysis of Non-linear Performance with Probability Distortion Xi-Ren Cao Department of Finance an Department of Automation, Shanghai Jiao Tong University, China, an the Institute of Avance Stuy, Hong Kong University of Science an Technology, China, Abstract: Human preference over ranom outcomes may not be as rational as shown in the expecte utility theory. Such an irrational (as a matter of fact, closer to reality) behavior can be moele by istorting the probability of the outcomes. Stochastic control of such a istorte performance is ifficult because ynamic programming fails to work ue to the time inconsistency. In this paper, we formulate the stochastic control problem with the istorte performance an show that the mono-linearity of the istorte performance, which claims that the erivative of the istorte performance equals the expecte value of the sample erivative uner a change probability measure, makes the graient-base sensitivity analysis suitable for optimization of the istorte performance. We erive the first orer optimality conitions (or the ifferential counterpart of the HJK equation) for the optimal solution. We use the portfolio allocation problem in finance as an example of application. Keywors: Probability istortion, mono-linearity, sensitivity-base optimization, perturbation analysis, portfolio allocation 1. INTRODUCTION The goal of a stanar control problem in finance is to optimize the expecte value of a utility function which represents the investor s ifferent satisfactions to ifferent outcomes. The expectation is taken with respect to the natural probability measure P an is a linear functional in the utility space (Ef(x)g(x) = Ef(x)Eg(x), with x representing the outcomes). However, people s satisfaction is not always linear (e.g., Ef(x)g(x) Ef(x) Eg(x)). In other wors, the stanar expecte utility theory cannot moel an explain this nonlinearity of people s behavior. Examples inclue lottery an insurance; in both cases, the expectation of the outcomes (benefit minus cost) are negative an people still buy them; that is, two ranom outcomes having the same expectation lea to ifferent satisfactions. This problem has been attracting consierable interests from the research community, especially in the financial sector. A ual theory is propose in Yaari (1987) to eal with this non-linear behavior. The essential point of the ual theory is that instea of istorting the outcomes by a utility function, it istorts the probability of the outcomes. Intuitively, the ual theory captures people s behavior that they usually enlarge the effects of rare events an iminish those of common events. Some excellent progress has been mae in portfolio optimization base on this performance moel with istorte probabilities, this research was supporte in part by NSF of China uner the grant , an the Research Grants Council, Hong Kong Special Aministrative Region, China, uner Grant No. HKUST11/CRF/1 an or the istorte performance for short Tversky an Kahneman (1992); Karatzas et al. (1991); He an Zhou (211); Jin an Zhou (28); Cao an Wan (213). These works, however, epen on the special structure of a financial market in which the stock prices are not controllable. The formulation of the optimization of the istorte performance an the approach to solve such an optimization problem in a general setting of stochastic control nee to be evelope. The main ifficulty is that ynamic programming fails in optimization of istorte performance. Because if a policy is optimal for a istorte performance when the process starts from time t, this policy is no longer optimal from time t onwar if the process starte from time t < t. This property is calle time-inconsistent; we nee to look for other approaches that o not require the time-consistent property. In the past ecaes, the author an his colleagues have been working on the sensitivity-base optimization approach Cao (27), which is an alternative to ynamic programming. Unlike ynamic programming, which requires time consistency, the sensitivity-base approach is simply base on a irect comparison of the performance of any two policies an therefore is not subject to time consistency. When the two policies uner comparison are infinitesimally close to each other, the approach leas to performance erivatives with respect to continuous parameters of policies. The performance an its erivatives can usually be measure on a sample path an therefore it is a sample-path base approach in the same spirit of perturbation analysis (PA) Ho an Cao (1991); Cao (27). It has been shown that many results by ynamic Copyright 214 IFAC 7785

2 Cape Town, South Africa. August 24-29, 214 programming can be obtaine with the sensitivity base approach Cao (27). On one sie, the stochastic control with istorte performance, which represents people s preference pattern over outcomes, cannot be solve by ynamic programming; on the other han, we have the sensitivity-base optimization approach, which oes not require the time-consistent property. It comes naturally to try apply this approach to the case with istorte performance. We have successfully applie this approach to the portfolio management problem in Cao an Wan (213), the results there rely heavily on the simple structure of the market ynamics: the price process of the stocks are not controllable. The goal of this paper is to formulate an stuy the optimization with istorte performance in a general setting of stochastic control, with the sensitivity-base approach. In Section 2, we efine the performance with probability istortion an formulate the stochastic control problem with istorte performance. In Section 3, we review some funamental properties of the istorte performance Cao an Wan (213). It has been shown that the istorte performance enjoys some sort of linearity, calle monolinearity, which forms the basis of the sensitivity-base approach applie to the problem. In Section 4, we present the general optimality conitions in a ifferential form (i.e., first orer optimality conition); Applying the perturbation analysis principle, an with the mono-linearity, we prove that erivative of the performance potential of an optimal polity forms a martingale with respect to the change measure. In Sections 5, we iscuss ynamic systems escribe by the stanar iffusion processes. We erive the equations for etermining the Raon-Nikoym erivatives of the change measure uner any policy w.r.t P, from which we obtain the first orer optimality conition in terms of the sample-erivatives of potentials an the system parameters. In this paper, we show that the mono-linearity of the istorte performance makes the graient-base sensitivity analysis suitable for optimization of the istorte performance. We prove that at an optimal policy, the sample erivative of the istorte performance is a martingale uner the change measure. First orer optimality conitions (or the ifferential counterpart of the HJK equation) are then obtaine for systems with iffusion state processes in terms of system parameters. These results cannot be obtaine by ynamic programming. 2. THE BASIC FORMULATION 2.1 The performance with istorte probability Consier a non-negative, a.s. finite ranom variable X efine on a probability space (Ω, F, P), with enoting a parameter. With the expecte utility theory, the objective is to maximize the expecte utility U(X ) : : U(X ) = PU(X ) > xx, (1) where enotes the expectation uner probability measure P. However, people s preference cannot always be measure by the expecte utility, e.g., they strongly islike isasters even if they happen rarely. This type of non-linear behavior has been wiely stuie in the finance an economics community. Wiely use is the ual theory propose in Yaari (1987), which moels the fact that people usually subconsciously enlarge the chance of rare events (wining a lottery or encountering a isaster) while iminishing the effect of common evens. Therefore, in the ual theory, we wish to maximize the following performance with a istorte probability (or calle the istorte performance ): η X := ẼPX := w{px > xx, (2) where w is a nonlinear istortion function, which is assume to be strictly increasing an analytical, with w() = an w(1) = 1. Compare with (2), the utility function U in (1) istorts the outcome values; this explains the meaning of ual. In finance, a convex istortion function is use by risk aversion investors, an a concave istortion function for risk taking ones. 2.2 The optimization problem Consier a stochastic system whose state S(t) follows the stanar iffusion process S(t) = α(t, S(t))t σ(t, S(t))W (t), t T, (3) with S() = S R N, where S(t) R N, an W (t) R M represents a Brownian motion efine on (Ω, F, P), α(t, S) R N, σ(t, S) R N M, an both α(t, S) an σ(t, S) are smooth functions on, t R N. Let F t, t T, be the filtration generate by W (t), t T ; we may take ω Ω as a sample path of the system states generate by a realization of W, enote as ω := {W (t), t T. Sometimes we a ω to inicate the epenence of a ranom variable on the sample path, e.g., S(t, ω). Let f(t, S), t, T, S R N, enote the time-epenent rewar function, an F (T, S(T )) enote the terminating rewar. The performance concerne is X (T ) = f(r, S(r))r F (T, S(T )), S() = S, (4) We wish to maximize the istorte performance Ẽ P X = w(h (x))x, (5) with H (x) = PX (T ) > x, subject to α(t, S) A, an σ(t, S) B, where A an B are given sets of n-imensional an n m imensional smooth functions, respectively; an we use to enote any parameter of α(t, S) an σ(t, S). In a stock market, we may have A = {all α(t, S) : α i (t, S i ) = µ i S i an B = {all σ(t, S) : σ i,j (t, S) = δ i,j σ i S i, with n = m an δ i,j =, if i j, an δ i,j = 1, if i = j; µ i an σ i, i = 1, 2,, N, are parameters. 3. FUNDAMENTALS We first review some funamental results in Cao an Wan (213) that motivate the stuy in this paper. 3.1 A weighte expectation form Let H (x) = PX > x be the ecumulative istribution function of a ranom variable X efine on (Ω, F, P), 7786

3 Cape Town, South Africa. August 24-29, 214 an G ( ) := H 1 ( ) be its left-continuous inverse. The istorte performance (2) can be reformulate as Ẽ P X = w(h (x))x = 1 G (w 1 (y))y = 1 z=z G (z) w(z) { = G (z) w(z) (6) where Z in (6) can be any uniform ranom variable on, 1 efine on (Ω, F, P). It is avantageous to take a special form of uniform ranom variable Z = H (X ) in (6). Throughout the paper we assume that X has no atom (the case with atoms is technically more involve an will be the topic for further research). In that case, G H (X ) = X, an (6) an (5) become w(z) Ẽ P X = X. (7) z=h (X ) 3.2 Change of measure We may use the ranom variable Π = w(z) in (7) z=h (X ) as a Raon-Nikoym erivative to efine another measure Q on Ω: Q P = w(z) (8) z=h (X ). Inee, w is strictly increasing an we have w(z) Π = z=h (X ) 1 w(z) = = w(1) w() = 1. Thus, (7) becomes Q Ẽ P X = X = E Q X. (9) P To efine a sample erivative, we use a function-like notation for a ranom variable: X = X (ω) := h(, ω). A sample erivative {X is efine as a erivative with ω = {W (t), t T fixe for both an : {X = {X (ω) = {h(, ω) X (ω) X (ω) = lim. Assumption 1. For any, {X = h(, ω) exist a.s.; H (x) is continuous ifferentiallable w.r.t. an x; {X X = x is continuous in x; there exists a ranom variable K with finite expectation, such that, X X w(z) K, a.s. (1) z=h (X ) for small enough. is boune in, 1, then (1) requires X uniformly ifferentiable at. This assumption to neee to ensure If w(z) the interchangeability of expectation an erivative, a well-known conition in perturbation analysis, an weaker conitions exist Ho an Cao (1991). Theorem 1. Uner Assumption 1, we have ẼX = E Q (X ) = {X w(z) z=h (X ) = E Q {X. (11) This important property is calle the mono-linearity in Cao an Wan (213). It shows that when we take erivatives, we may only change the measure at one en of the erivative irection; this property makes it possible to use the sample erivative an is the founation for our analysis in optimization of istorte performance. 4. OPTIMIZATION OF DISTORTED PERFORMANCE: AN OVERVIEW 4.1 Stochastic control revisite When w 1, the problem becomes a stanar control problem. Let us briefly revisit the stochastic control problem with a sensitivity-base view an then erive the first orer optimality conition. On any sample path enote by ω, we efine the sample performance η (ω) = f (r, S (r), ω)r F (T, S (T ), ω), (12) an the sample potential function g (t, ω) = t f (r, S (r), ω)r F (T, S (T ), ω). (13) Then η (ω) = g (, ω) an the system performance is η (S) = {η (ω) S () = S, an the performance potential function is g (t, S) = {g (t, ω) S (t) = S, t T, S. (14) We can easily prove that g (t, S) satisfies the Poisson equation A g (t, S) f (t, S) =, t T, S, (15) with g (T, S) = F (T, S), an the infinitesimal generator A is efine with any smooth function h(t, S) (with subscript omitte): Ah(t, s) = τ E { = E h(τ, S(τ)) S(t) = S { t h(t, S(t)) S(t) = S. (16) We have the performance erivative formula (which can be erive by Dynkin s formula, see Cao et.al (211)) η (S) = ġ (, S) = {ġ (, ω) S () = S { = Ȧg f (t, S (t))ts () = S (17), 7787

4 Cape Town, South Africa. August 24-29, 214 where η, etc, enote the partial erivative with respect to. By setting η (S) =, we obtain the first orer optimality conition as Cao et.al (211) Ȧg f (t, S) =, t T, S. (18) This is the ifferential version of the HJB equation, an can also be obtaine by taking erivative of the stanar HJB equation. On the other han, taking erivatives with respect to on both sies of (15) yiels Ȧg A ġ f (t, S) =, t T, S. Thus, the performance erivative formula (17) becomes { η (S) = A ġ (t, S (t))ts () = S, (19) an the first orer optimality conition is A ġ (t, S) =, t T, S. (2) In aition, we nee a sample path base conition. From (2), we have 1 { A ġ (t, S) = lim τ τ ġ (t τ, S (t τ)) S ġ (t, S (t)) (t) = S { = t ġ (t, ω) S (t) = S, t T, S. Thus, the first orer optimality conition takes the form {ġ (t, ω) S (t) = S = { ġ (τ, ω) S (t) = S =, t T, S. (21) 4.2 The first orer optimality conition for istorte performance Now, we turn to the optimization of istorte performance. Define ξ T (ω) = w(z) (22) z=h (η (ω)), an ξ t = ξ T F t, t T, with ξ = ξ T F = 1. (23) By efinition, ξ t is a martingale uner P. We wish to optimize the istorte performance efine as (cf. (7)) η (S) = g (, ω)ξ T (ω) S () = S = E Q η (ω) S () = S, (24) with Q be a measure efine by the Raon-Nikoym erivative Q w(z) (ω) = = ξ T (ω). (25) P z=h (η (ω)) By mono-linearity (11) in Theorem 1, we have η (S) = E Q η (ω) S () = S. The erivative of the istorte performance (19) is now { η (S) = E Q A Q ġq (t, S (t))ts () = S, (26) in which A Q enotes the infinitesimal generator uner measure Q, an g Q (t, S (t)) is efine in (14) uner measure Q. Therefore, we obtain the first orer optimality conition for the istorte performance: If is an optimal policy an Q is corresponing istortion measure, then A Q ġq (t, S) =, t T, S. (27) Before turning to the sample path base conition, we nee to reform ξ T (ω). From (12), we have η (ω) = f(r, S (r), ω)r t f(r, S (r), ω)r F (T, S (T ), ω) = R (t, ω) g (t, S (t), ω), with R (t, ω) an g (t, S (t), ω) enoting the two terms in the sum. R (t, ω) satisfies R(t, ω) = f(t, S(t), ω)t. (28) Set ẇ(z) = w(z), then ξ T = ẇ{h η (ω) = ẇ{h R (t, ω) g (t, S (t), ω). By the Markov property of S(t), given R (t, ω) = R(t) an S (t, ω) = S(t), ξ t = ξ T F t = {ẇ{h R (t, ω) g (t, S (t), ω) F t is a function of R(t) an S(t). We enote it as ξ t (ω) = ξ t (S(t, ω), R(t, ω)). (29) Now, by (21), the sample path base conition is {E Q ġ (τ, ω) S (t) = S =, t T, S. (3) For τ t, we have E Q ġ (τ, ω) S (t) = S, R (t) = R = ġ (τ, ω)ξ T (ω) S (t) = S, R (t) = R = ġ (τ, ω)ξ τ (S(τ, ω), R(τ, ω)) S (t) = S, R (t) = R. Thus, (3) becomes { ( ġ (τ, ω)ξ τ (S(τ, ω), R(τ, ω)) ) S (t) = S, R (t) = R S (t) = S =, an we have the optimality equation ġ (t, ω)ξ t (S (t), R (t)) S (t) = S, R (t) = R =, or S { ġ (t, ω)ξ t (S (t), R (t)) () = S =, (31) t T, (32) This equation implies the following theorem: 7788

5 Cape Town, South Africa. August 24-29, 214 Theorem 2. For an optimal policy, the process ġ (t, ω)ξ t (S (t, ω), R (t, ω)) is a martingale with respect to F t an probability measure P. In particular, when w 1, it is the stanar stochastic control problem an for an optimal policy, ġ (t, ω) is a martingale with respect to P. Globally, we have S { ġ (t, ω)ξ t (S (t), R (t)) () = S =, t T. (33) In aition, the erivative of the istorte performance (26) becomes η (S) = {ġ (t, ω)ξ t (S (t), R (t)) S () = S. (34) Finally, if f(t, S), then we have R(t, ω) = an (29) becomes ξ t (ω) = v(t, S(t, ω)). (35) In the next two sections, we will erive optimality conitions expresse explicitly in system parameters. The potential-base optimality conition is base on (27); an the sample erivative base conitions are base on (31) or (32). 5. POTENTIAL-BASED OPTIMIZATION CONDITION 5.1 Determining the martingale ξ t By the martingale representation theorem (see Karatzas an Shreve (1991), Problem ), there exists an F t - aapte an R n -value process ζ(t) = (ζ 1 (t),..., ζ M (t)) with ξt 2 ζm(t)t 2 <, such that ξ t = ξ T m=1 m=1 ξ r ζ m (r)w m (r), t T (36) hols a.s. P. Therefore, ξ t = ξ t ζ(t)w (t), (37) an from which, we get ξ t (ω) = exp { 1 2 ζ 2 (s)s ζ(s)w (s). (38) We first assume f(t, s) an use the form in (35) ξ t (ω) = v(t, S(t, ω)). Since ξ t is a martingale, we have A t ξ t =. Thus, the function v(t, S) satisfies the following Poisson equation N t v(t, S) α i (t, S(t)) v(t, S) (σσ T 2 ) ij v(t, S) =, (39) S j with the bounary conition v(, S) = ξ T F = 1, (4) an v(t, S) = w(z) (41) z=h (η (ω)), where η (ω) = F (T, S). On the other han, by the Ito rule, we have ξ t = v(t, S(t)) = t v(t, S)t N σ ij (t, S(t))W j (t) v(t, S)α i (t, S(t))t (σσ T 2 ) ij v(t, S)t. S j Comparing this equation with (39), we have ξ t = = v(t, S) M σ ij (t, S(t))W j (t) N σ ij (t, S(t)) v(t, S) W j (t). Comparing this with (37), we have σ ij (t, S(t)) v(t, S) = v(t, S(t))ζ j (t), j = 1, 2, M. (42) Therefore, ζ j (t), j = 1, 2,, M, can be obtaine from (41), with v(t, S) being the solution to (39) an (4). When f, we take the R(t) in (28) as a system state an a (28) to the system equations: { R(t) = f(t, S(t))t, (43) S(t) = α(t, S(t))t σ(t, S(t))W (t). Set S (t) = R(t) an α (t, S) = f(t, S), σ,i (t, S) = σ j, (t, S) =. This equation looks the same as a regular system with one more imension. With this notation, we may enote ξ t (ω) = v(t, S(t)), which has the same form as (35), with S(t) = (S (t), S 1 (t),, S N (t)) T. With this moification, all the results (36-41) carries over except that in the bounary conition (4), we have η (ω) = F (T, S), R = S (T ). Furthermore, with this moification all the other results for f apply to the case f. Therefore, in what follow we only work on the form (35). 5.2 The optimality conition for (24) Now, we calculate the sample erivative 7789

6 Cape Town, South Africa. August 24-29, 214 E{ġ t, S (t)ξ t S (t) S (t) = S = E { {ġ t, S (t)ξ t S (t) S (t) = S. in the optimality conition (??). First, we have where {ġ t, S (t)ξ t S (t) = {ġ t, S (t)ξ t S (t) ġ t, S (t){ξ t S (t) {ġ t, S (t){ξ t S (t), (44) ġ (t, S (t)) = {E P f(r, S (r), ω)r t F (T, S (T ), ω) S (t) = S. By Ito rule, we have ġ (t, S (t)) = tġ(t, S (t))t Therefore, α i (t, S(t))t ġ (t, S (t)) σ i,j (t, S(t))W j (t) (σσ T 2 ) i,j (t, S(t)) ġ (t, S (t))t. S j ( {ġ t, S (t)ξ t S (t) S (t) = S ) = { tġ(t, S)t ġ (t, S) α i (t, S)t (σσ T 2 ) i,j (t, S) ġ (t, S) ξ,t (S)t. S j From (37), we have (ġ t, S (t){ξ,t S (t) S (t) = S ) =, an ( {ġ t, S (t){ξ t S (t) S (t) = S ) { M = ġ (t, S) σ i,j (t, S)ζ j,,t (S) ξ,t (S)t. Therefore, the first orer optimality conition (31) is { ) S ( ġ (t, S (t))ξ t (S (t)) (t) = S t { = tġ(t, S) ġ (t, S) α i (t, S) (σσ T 2 ) i,j (t, S) ġ (t, S) S j ( ġ (t, S) M =, t T, S. ) σ i,j (t, S)ζ j,,t (S) ξ,t (S). Because ξ,t (S) >, so the first orer optimality conition (31) becomes tġ(t, S) ġ (t, S) α i (t, S) (σσ T 2 ) i,j (t, S) ġ (t, S) S j ( ġ (t, S) M ) σ i,j (t, S)ζ j,,t (S) =, t T, S. (45) Conition (44) can also be obtaine irectly from (27). From the Girsanov theorem, with (38), uner measure Q, the rift of the Brownian motion has to change accoring to ζ(t): W Q (t) = W (t) ζ(t)t. Uner measure Q, the system equation has to be moifie accoring to the above equation: S(t) = α(t, S(t)) σ(t, S(t))ζ(t)t σ(t, S(t))W Q (t), (46) with S() = S. Applying A Q accoring to the system equation (45) to ġ (t, S) yiels (44). REFERENCES X. R. Cao. Stochastic Learning an Optimization, Springer, New York, 27. X. R. Cao an X. W. Wan. Sensitivity Analysis of Nonlinear Behavior with Distorte Probability, Mathematical Finance, submitte, 213. X. R. Cao, D. X. Wang, T. Lu, an Y. F. Xu. Stochastic Control via Direct Comparison, Discrete Event Dynamic Systems: Theory an Applications, Vol. 21, pages 11-38, 211. Y. C. Ho an X.R. Cao. Perturbation Analysis of Discrete- Event Dynamic Systems, Kluwer Acaemic Publisher, Boston, M. E. Yaari. The ual theory of choice uner risk. Econometrica, 55(1), pages , I. Karatzas, J. Lehoczky, S. Shreve, an G. Xu. Martingale an uality methos for utility maximization in an incomplete market. SIAM Journal on Control an Optimization, 29(3), pages 72-73, 1991 A. Tversky an D. Kahneman. Avances in prospect theory: Cumulative representation of uncertainty. Journal of Risk an Uncertainty, 5, pages , X. D. He an X.Y. Zhou (211) Portfolio choice via quantiles, Mathematical Finance, 21(2), pages , 211. H. Jin an X.Y. Zhou. Behavioral portfolio selection in continuous time. Mathematical Finance, 18, pages , 28bv. 779