Optimal Energy Commitments with Storage and Intermittent Supply

Transcription

1 Submitte to Operations Research manuscript OPRE Optimal Energy Commitments with Storage an Intermittent Supply Jae Ho Kim Department of Electrical Engineering, Princeton University, Princeton, NJ 08540, Warren B. Powell Department of Operations Research an Financial Engineering, Princeton University, Princeton, NJ 08540, We formulate an solve the problem of making avance energy commitments for win farms in the presence of a storage evice with conversion losses, mean-reverting price process, an an auto-regressive energy generation process from win. We erive an optimal commitment policy uner the assumption that win energy is uniformly istribute. Then, the stationary istribution of the storage level corresponing to the optimal policy is obtaine, from which the economic value of the storage as the relative increase in the expecte revenue ue to the existence of storage is obtaine. Key wors : Markov Decision Process, Dynamic Programming, Energy History : This paper was first submitte on September 4, 2009 an has been revise an re-submitte on May 27, 2010, an on January 6, Introuction The emphasis on renewables, such as the goal set by the Department of Energy to have 20 percent of electric power from win by 2030, has raise the importance of efficiently managing win, an unerstaning the factors that affect the cost of using win. Currently, win energy accounts for a small fraction in the market an the gri operators allow the win energy proucers to eliver any amount of energy they prouce at a given time. However, as the share of win energy in the market grows, such a policy will become impractical, an gri operators will nee to make commitments on the amount of win energy that will be elivere in avance. Unfortunately, making commitments is complicate by the inherent uncertainty of win. This uncertainty can be mitigate by the presence of storage, which also introuces the imension of losses ue to the conversion neee to store an retrieve energy. 1

2 2 Article submitte to Operations Research; manuscript no. OPRE We aress the problem of making a commitment at time t to eliver energy from win uring the time interval t, t + 1). The moel is most easily applie in the hour-ahea market, although it can be use in an approximate fashion in the ay-ahea market. Energy storage has long been recognize as an important technology for smoothing the variability of win Castronuovo an Lopez 2004), Korpaas, et al. 2003), Garcia-Gonzalez, et al 2008), Brunetto an Tina 2007), Ibrahim, et al. 2008)). We assume that we store energy when the available energy from win excees the commitments we have mae, but we may incur significant conversion losses. The problem has cosmetic similarities with classical inventory problems storing prouct to meet eman), but with some funamental ifferences. Inventory problems are typically trying to control the supply of prouct to meet an exogenous eman Axsäter 2000), Zipkin 2000)). In our problem, we have exogenous supply energy generate from win) to meet eman by making avance commitments. This problem is similar to the reservoir management problem which is characterize by ranom rainfall see Nanalal an Bogari 2007) for an excellent review of ynamic programming moels for reservoir management). Our problem is istinguishe by the nee to make avance commitments, along with our interest in a simple, analytical solution that can be use in economic moels. In this paper, we erive an optimal policy for making energy commitments from win in the presence of an energy storage evice. We then use this policy to stuy the economics of storage capacity in this setting. Given the richness of the application, we analyze a stylize version of the problem, which allows us to erive the optimal policy in a simple, analytic form. Our moel captures some important imensions of the real problem such as the storage capacity constraints, storage conversion losses an a mean-reverting process for real-time electricity prices. At the same time, we make a number of simplifying assumptions. Some of these inclue: We assume that we are a small player in a large market, making it possible to sell all of the energy we prouce as long as we make avance commitments. In aition, we assume that if the energy from win plus what is available in storage) falls below our commitment, that we can make up the entire shortfall using the current spot price.

3 Article submitte to Operations Research; manuscript no. OPRE If the moel is applie in a ay-ahea market, we ignore the ability to make hour-ahea ajustments. We capture storage capacity an conversion losses, but we otherwise ignore the physics of energy storage, such as the relationship between the rate of storage an storage capacity, an the impact of full ischarges on battery life. We assume mean-reverting electricity prices an stationarity in the errors in win forecasts. Our analytical moel assumes that win follows a uniform istribution, although we then quantify this error in experimental comparisons using actual win patterns. More realistic moels require an algorithmic solution. A goal of our research is a simple policy that can be use in economic moels without requiring the complex machinery of stochastic optimization algorithms. The goal of a win farm operator is to maximize the cumulative profit over time by computing the amount of electricity to commit to sell uring the time interval t, t + 1) at each time t. Brown an Matos 2008), Brunetto an Tina 2007), Castronuovo an Lopez 2004), an Korpaas, et al. 2003) attempt to solve the problem by solving a eterministic optimization problem given a particular sample path over a finite horizon an then averaging the results over the sample paths. The sample paths are rawn from a fixe T + 1)-imensional istribution escribing the electricity generate from the win farm uring the time interval t, t + 1) for each t = 0, 1,..., T. However, this approach oes not prouce a vali, amissible policy. In practice, we nee a policy that allows the win farm operator to compute at time t the amount of electricity to commit to sell uring the time interval t, t + 1) base on the state of the environment at time t. The objective of this paper is to fin such a policy an analyze it. The contributions of this paper are as follows. 1) We erive an analytical expression for an optimal policy, an the value of storage, for a stylize moel of an energy storage process in the presence of intermittent generation requiring avance commitments. 2) We establish assumptions on the electricity price an the istribution of win, size of the storage, an the ecision epoch intervals that allow us to erive the optimal policy for energy commitment in a close form an

4 4 Article submitte to Operations Research; manuscript no. OPRE explain the implications of those assumptions. 3) Uner those assumptions, we erive the optimal policy for avance energy commitment in a simple, analytical form, when we have a storage with an arbitrary roun-trip efficiency, an when electricity prices are mean-reverting. The optimal policy obtaine uner such assumptions resembles the optimal policy for the well-known newsvenor problem Khouja 1999), Petruzzi an Daa 1999)). 4) We obtain the stationary istribution of the storage level corresponing to the optimal policy, from which we fin the economic value of the storage as the relative increase in the expecte revenue ue to the existence of storage. 5) We test our policy using win energy generate from truncate Gaussian istributions an emonstrate that the error introuce by assuming a uniform istribution for win is reasonably small. This paper is organize as follows. In 2, we moel the win energy storage problem as an MDP with continuous-state an control variables. In 3, we present our assumptions an the structural properties of the optimal value function of the MDP. In 4, the optimal policy for the infinite horizon problem for a storage with general roun-trip efficiency is obtaine. Then, the stationary istribution of the storage level corresponing to the optimal policy is obtaine, from which the economic value of the storage as the relative increase in revenue ue to existence of storage, is erive. In 5, we compute the economic value of storage using the win spee ata obtaine from the North American Lan Data Assimilation System NLDAS) project Cosgrove, et al. 2003)), an the electricity price ata provie by a utility company. In 6, we summarize our conclusions. 2. The moel Operating a win farm epens on two markets: the electricity spot market an the regulating market. We sell to the spot market an pay a penalty when we fail to meet our commitment. The gri operator buys energy from the regulating market when we fail to meet our commitment. In the spot market, the energy proucers make their commitments to eliver sell) electricity in avance while the regulating market is a marketplace for reserve energy in which the proucers have the ability to sell electricity on a shorter notice than the spot market Korpaas, et al. 2003),

5 Article submitte to Operations Research; manuscript no. OPRE MacKerron an Pearson 2000), Morthorst 2003)). As a win farm operator, when the electricity prouction excees our expectation an we have an excess amount of electricity left over after fulfilling the contractual commitment, we store the excess amount. On the other han, when the electricity prouction falls too short to meet the contractual commitment, we have to pay a premium, a penalty for failing to meet the commitment, while the proucers in the regulating market make up for the gap. Therefore, if we commit too much, we can actually lose money. We have revenues from our sale on the spot market an costs from tapping into the regulating market when we fail to meet our commitment for elivery on the spot market see Chapter 16 of MacKerron an Pearson 2000) for a etaile exposition of the market system). At each time t, the market participants submit their bi for the supply an eman for electricity that must be elivere uring the time interval t, t + 1). The market overseer collects the biing information an etermines the spot market an the regulating market price for the time interval t, t + 1) shortly after the participants submit their bis. Therefore, as a win farm operator, we o not know what the prices will be when we are making our commitments. We make the following assumptions. First, we assume that at each time t, we have a probability istribution of the electricity we will generate uring the time interval t, t + 1). Secon, we assume that we are a small participant in the market such that the market can always absorb our supply an the effect of our biing on the expecte spot market an the regulating market prices of the electricity is negligible. Then, the prices can be treate as exogenous variables an we only nee to etermine the amount of electricity to commit to sell. Thir, we assume that the spot market price of the electricity is mean-reverting an the ratio of the expecte spot market price over the expecte regulating market price is always less than the roun-trip efficiency of our storage with the iscount factor. Otherwise, the cost of using the storage, which can be measure by the conversion loss, will be greater than the expecte cost of tapping into the reserve energy in the regulating market, negating the purpose of using a storage evice in the first place. The thir assumption is crucial in maintaining the concavity of the optimization problem.

6 6 Article submitte to Operations Research; manuscript no. OPRE System Parameters R max = upper limit on the storage. unit: storage energy capacity unit) = coefficient use to convert the generate electricity to potential energy in the storage. unit: storage unit / electricity unit) ρ E = coefficient use to convert the potential energy in the storage to electricity. unit: electricity unit / storage unit) Note that 0 < ρ E < 1, where ρ E enotes the roun-trip efficiency. Throughout this paper, 1 ρ E is referre to as the conversion loss from storage. ρ E is aroun for most of the existing storage systems Sioshanshi, et al. 2009)). µ p = mean of the spot market price of the electricity. unit: ollar / electricity unit) σ p = stanar eviation of the change in spot market price of the electricity. unit: ollar / electricity unit) κ = mean-reversion parameter for the spot market price of the electricity. κ is proportional to the expecte frequency at which the spot market price crosses the mean per unit time. unit: 1 / time unit) τ = time interval between ecision epochs. m = slope of the penalty cost for over-commitment. b = intercept of the penalty cost for over-commitment. unit: ollar / electricity unit) That is, when the spot market price of the electricity is p t, the penalty for over-commitment is mp t + b. µ Y = mean of the electricity generate from the win farm per unit time. unit: electricity unit / time unit) σ Y = stanar eviation per unit time of the electricity generate from the win farm. unit: electricity unit / time unit) γ = iscount factor in the MDP moel. 0 < γ < 1.

7 Article submitte to Operations Research; manuscript no. OPRE State Variables Let t N + be a iscrete time inex corresponing to the ecision epoch. The actual time corresponing to the time inex t is t τ. R t = storage level at time t. 0 R t R max, t. Y t = electricity generate from the win turbines uring the time interval t 1, t). Y t 0, t. p t = spot market price for electricity elivere uring the time interval t 1, t). p t 0, t. W t = Y t ), p 1 t t t) = exogenous state of the system. S t = R t, W t ) = state of the system at time t Decision Action) Variable x t = amount of electricity we commit to sell on the spot market uring the time interval t, t + 1) etermine by signing the contract at time t. x t 0. Since we are making an avance commitment, x t is not constraine by R t. The lack of an upper boun on x t inicates that we are a small player in the market an hence there will always be enough eman in the market to absorb our supply as long as we are making an avance commitment Exogenous Process ŷ t = noise that captures the ranom evolution of Y t. Specifically, M 1 Y t+1 = µ Y τ + α i Y t i µ Y τ) + ŷ t+1, 1) i=0 for some orer M an coefficients α i for 0 i M 1. ŷ t ) t 1 an Y t ) t 1 must be proportional to τ. p t = noise that captures the ranom evolution of p t. Specifically, we use a iscrete-time version of the Ornstein-Uhlenbeck process: p t+1 p t = κ µ p p t ) τ + p t+1. Let Ω be the set of all possible outcomes an let F be a σ-algebra on the set, with filtrations F t generate by the information given up to time t : F t = σ S 0, x 0, Y 1, S 1, x 1, Y 2, S 2, x 2,..., Y t, S t, x t ).

8 8 Article submitte to Operations Research; manuscript no. OPRE P is the probability measure on the measure space Ω, F). Throughout this paper, a variable with subscript t is unknown ranom) at time t 1 an becomes known eterministic) at time t. In other wors, a variable with subscript t is F t -measurable. We have efine the state of our system at time t as all variables that are F t -measurable an neee to compute our ecision at time t Storage Transition Function R max, if R t + Y t+1 x t ) R max. R t + Y t+1 x t ), if x t < Y t+1, R t + Y t+1 x t ) < R max. R t+1 = R t 1 ρ E x t Y t+1 ), if Y t+1 x t < ρ E R t + Y t+1. 0, if x t ρ E R t + Y t+1. If Y t+1 excees the commitment x t, we store the excess amount Y t+1 x t with a conversion factor,. If Y t+1 is less than x t, the potential energy in the storage must be converte into electricity with a conversion factor, ρ E, to fulfill the gap, x t Y t+1. If the amount of electricity generate uring the time interval t, t + 1) plus the electricity that can be obtaine by converting the potential energy in the storage is not enough to cover the contractual commitment, we eplete our storage an we have to pay for the gap. It is important to note the ifference between the storage transition function shown above an the transition functions that generally appear in traitional inventory management an resource allocation problems Axsäter 2000), Zipkin 2000)). Unlike many of the transition functions that appear in traitional problems, here R t+1 is not a concave or convex function of x t or R t, which makes the concavity of the optimization problem not obvious. 2) 2.6. Contribution Revenue) Function The profit we make uring the time interval t, t + 1) is given by { p Ĉ t+1 = t+1 x t, if x t < ρ E R t + Y t+1. p t+1 x t mp t+1 + b) x t ρ E R t + Y t+1 )], if x t ρ E R t + Y t+1. p t+1 x t is the profit we earn by elivering x t amount of electricity to the market uring the time interval t, t + 1), an mp t+1 + b is the penalty we pay in the case of over-commitment. Assume m γ ρ E an b γ ρ E µ p. 3)

9 Article submitte to Operations Research; manuscript no. OPRE Then, the cost of using the storage, which can be measure by the conversion loss, is less than the cost of over-commitment. Otherwise, for the purpose of maximizing the revenue, there will be no reason to use a storage in the first place. concavity of the stochastic optimization problem. This affine penalty is sufficient to ensure the Note that these lower bouns on the penalty factor are unfavorable assumptions - they make the environment in which we operate more averse an lea to a more conservative policy. If we have to operate in an environment where the above assumptions o not hol, the optimal policy erive in this paper uner the above assumptions may not be optimal in maximizing revenue, but it shoul still be robust with limite risk - we lose less money than expecte in the case of over-commitment. Define where ] CS t, x t ) := E Ĉt+1 S t, x t = µ p + 1 κ τ) p t µ p )] x t m F t y)y b 0 y x t ρ E R t F t y) = P Y t+1 y F t ]. 0 y x t ρ E R t F t y)y, 4) C ) is known as the contribution, or the rewar function. See 1 in the e-companion to this paper for the erivation of 4) Objective Function Let Π be the set of all policies. A policy is an F t -measurable function X π S t ) that escribes the mapping from the state at time t, S t, to the ecision at time t, x t. For each π Π, let T ] G π t S t ) := E γ t t CS t, X π S t )) S t, 0 t T, t =t where 0 < γ < 1 is the iscount factor an T inicates the en of the horizon. The objective, then, is to fin an optimal policy π = π that satisfies for all 0 t T. G π t S t ) = sup G π t S t ), π Π

10 10 Article submitte to Operations Research; manuscript no. OPRE Main Assumptions an Structural Result The main contribution of this paper is the close form representation of the optimal policy for avance intermittent energy commitments that also allows us to express the value of the energy storage in a close form. In orer to achieve the results, we nee assumptions on the probability istribution of the spot market electricity price an win energy, limit on the storage size, an the ecision epoch intervals Electricity Price an Win Energy First, we assume that p t ) t 0 an ŷ t ) t 1 are inepenent in Ω, F, P). It is well-known that the price of the electricity mainly epens on the eman as well as the main source of energy that is controllable; for example, electricity generate from coal plants. It is fairly reasonable to assume that the fluctuation in the electricity price is not significantly influence by the fluctuation in the uncontrollable an unpreictable energy supply from our win farm, especially if we are a small player in the market. In most cases, intermittent energy plays a minor role in the electricity markets, anyway. Next, assume p t ) t 0 are i.i. with istribution N 0, σ 2 p). Then, pt ) t 0 is a stanar meanreverting process an E p t+n F t ] = µ p + 1 κ τ) n p t µ p ), n, t N +. 5) It is common to use a mean-reverting process to moel electricity prices, as shown in Eyelan an Wolyniec 2003).Similarly, assume ŷ t ) t 1 are 0-mean an i.i. with stanar eviation σ Y τ. Then, in most cases, the istributions of ŷ t ) t 1 are assume to be truncate Gaussian with mean 0 an stanar eviation σ Y τ. However, in this paper, we assume that ŷ t ) t 1 are uniformly istribute with mean 0 with stanar eviation σ Y τ. Assuming that ŷ t ) t 1 are uniformly istribute allows us to explicitly compute various expectations that are neee to erive the optimal policy in a close form. Since a truncate Gaussian istribution is boune, as long as we match the mean an the variance, a uniform istribution can be a statistically robust substitute

11 Article submitte to Operations Research; manuscript no. OPRE for the truncate Gaussian istribution in the context of optimizing a value function. This fact is emonstrate in 5 where we conuct numerical experiments in which we apply the optimal policy erive uner the assumption of uniformly istribute ŷ t ) t 1 to the ata generate from truncate Gaussian istributions. Then, given F t, Y t+1 U θ t, θ t + β), where β := 2 3σ Y τ an M 1 θ t := µ Y τ + α i Y t i µ Y τ) β, t. 6) 2 The cumulative ensity function CDF) of Y t+1 compute at time t is given by 0, if y < θ t y θ F t y) = P Y t+1 y F t ] = t if θ β t y θ t + β 1, if y > θ t + β i=0 The expecte contribution function C ) is not inexe by t because the CDF F t ) is etermine by θ t, which is a eterministic function of S t. The expecte contribution is completely etermine by S t an x t. However, it is important to note that θ t an β o not necessarily have to be efine as shown above. The results obtaine in this paper are applicable as long as we use a forecasting moel that preicts that the electricity prouce uring the time interval t, t + 1) is uniformly istribute given F t Size of the Storage Next, we nee an assumption on the size of the storage. Since the electricity price is mean-reverting, if we have an infinitely large storage, a naive policy that stores the energy when the expecte spot market price is less than some fixe price an commits to sell the energy in storage plus the energy we are certain to prouce when the spot market price is greater than some fixe price, will be a riskless arbitrage policy. Arbitrage here means that there is zero probability of losing money ue to over-commitment or losing energy ue to the storage being full. There is always a significant conversion loss. Such a case is comparable to traing a stock whose price is mean-reverting. In reality, a storage with reasonably goo roun-trip efficiency that can be charge an ischarge

12 12 Article submitte to Operations Research; manuscript no. OPRE in a short amount of time will be expensive to buil an maintain, an we nee an intelligent way of etermining the appropriate size of the storage. We propose that the size of the storage be etermine in comparison to ν, given by: ] σ Y m 1 ν := κ 2 3 min m, b. 7) b + ρ E γµ p It is obvious that as the penalty factors m an b become larger, we nee to allow for a larger storage since our commitment level will be more conservative an we will en up storing more energy. Also, if the roun-trip efficiency of the storage ρ E is small, we must allow for a larger storage in orer to compensate for the energy that will be lost in conversion. Next, since γµ p is the iscounte expecte spot market price of the electricity, if γµ p is small, we nee to allow for a larger storage since our commitment level will be more conservative. What makes ν interesting is the term σ Y /κ. Recall that κ is proportional to the expecte number of times the price crosses the mean per unit time. Then, 1/κ is proportional to the expecte amount of time between two consecutive crossings. Therefore, σ Y /κ is proportional to the volatility in the win energy that is prouce while the spot market price completes a cycle. Since R max etermines our ability to accumulate energy while the price moves, we must allow for a larger storage when σ Y /κ gets larger. If R max =, we can implement an arbitrage policy, as explaine above. If R max ν, we must implement a more active, risk-taking policy that consiers the movement of the price towars the mean but not count on the price reaching a esirable level within a esirable amount of time. The mile regime in which ν < R max < will eman the most complicate policy that mixes risk-taking with arbitrage. Fining the optimal policy in this mile regime will be an interesting research topic, but it is beyon the scope of this paper. For this paper, we assume ] σ Y m 1 R max κ 2 3 min m, b. 8) b + ρ E γµ p 8) is necessary in orer to erive 10) shown in the next section, which in turn is necessary to prove lemma 18) that is use to erive the marginal value function in a close form. However,

13 Article submitte to Operations Research; manuscript no. OPRE even though 8) is impose for mathematical convenience, numbers come out reasonable, as shown in 5. If we use real ata to obtain σ Y, κ, µ p, ρ E an use m an b that satisfy 3), if we let R max σ Y = 1.7, for example, 8) is satisfie. That is, we can have the size of R max in the same orer of magnitue of the stanar eviation in win energy. Having a storage of limite size allows us to obtain the optimal policy in a close form an provie us with various insights, as is shown in 4. Moreover, before investing a significant amount of capital to buil a large storage, it is reasonable to assume that win farm operators will start with a small storage, stuy its effects, an then subsequently make the investment for aitional storage. This paper erives the optimal policy for energy commitment an the corresponing value of the storage when the storage is small. As will be sown in 4, the optimal policy uner the assumption 8) will still epen on the mean of the electricity price an how far the price is away from the mean. However, the optimal policy will be base on the premise that the storage is not large enough to allow us to avoi the risk of over-commitment by waiting for the price to rise without facing the risk of losing energy ue to the storage being full. Thus, 8) forces us to always balance the risk of over-commitment an the risk of uner-commitment. We not only want to avoi paying the penalty for over-commitment, but we also want to avoi committing too little an lose energy ue to conversion an the storage being full. Suppose we have a large storage evice an 8) is violate, but we choose to implement the policy erive in this paper that is optimal uner the assumption of small storage, anyway. Then, the cost of over-commitment will not change, but the risk of uner-commitment will be smaller than expecte because we are less likely to lose energy ue to the storage being full. Therefore, the optimal policy erive in this paper will still be robust when the assumption 8) oes not hol Decision Epoch Interval Finally, we nee an assumption on how often we make our commitment ecisions. We can re-arrange the terms from 8) to obtain:

14 14 Article submitte to Operations Research; manuscript no. OPRE max R max m ρ E γ) 2 3 m 1) σ Y R max ρ E γκ, R maxb 2 3 σ Y b R max ρ E γκµ p ] 1 κ. We assume that the time interval τ between our ecision epochs satisfies the following: max R max m ρ E γ) 2 3 m 1) σ Y R max ρ E γκ, R maxb 2 3 σ Y b R max ρ E γκµ p ] τ 1 κ. 9) 9) ensures that the price always moves towar the mean in expectation, but oes not overshoot an move pass the mean in expectation. The lower boun can be re-arrange to be written as m 1 R max β min m ρ E γ 1 κ τ), b b + ρ E γκ τµ p ]. 10) Since we have a limit on the size of our storage as shown in our assumption 8), if τ is too large, the amount of electricity that is prouce between our ecisions can be too large an we are likely to lose energy ue to the storage being full. 9) gives us a reasonable ecision epoch time interval τ Structural Results In this section, we show some structural results of the value function. Let V π t S t ) be a function that satisfies V π T S T ) = CS T, X π T S T )), V π t S t ) = CS t, X π t S t )) + γe V π t+1s t+1 ) S t ], 0 t T 1. Then, V π t S t ) = G π t S t ), 0 t T. For 0 t T, let V t S t ) satisfy the following: V T S T ) = max x R + CS T, x), V t S t ) = max x R + {CS t, x) + γe V t+1 S t+1 ) S t, x]}, 0 t T 1. V t S t ) is known as the value function. Accoring to Puterman 1994), V t S t ) = G π t S t ), 0 t T. Denote V x t S t, x) := E V t+1 S t+1 ) S t, x], 0 t T.

15 Article submitte to Operations Research; manuscript no. OPRE The augmente value function V x t S t, x) is an example of a Q-factor. Let Then, x t := arg max {CS t, x) + γv x t S t, x)} = X π S t ), 0 t T. x R + V t S t ) = max x R + {CS t, x) + γv x t S t, x)} = CS t, x t ) + γv x t S t, x t ), 0 t T. At the en of the horizon, we can show that R T V T S T ) = ρ E µ p + 1 κ τ) p T µ p )] 11) an hence 2 R 2 T V T S T ) = 0. 12) See 2 in the e-companion to this paper for the erivation of 11) an 12). Now that we have efine the value function, we present its structure. The structural results are mainly attributable to the storage transition function an the contribution function, an they follow from three of the aforementione assumptions: ŷ t ) t 1 an p t ) t 1 are inepenent, p t ) t 1 is mean-reverting as shown in 5), an Then, 0 t T 1, we have: m γ an b γ µ p. ρ E ρ E Structural Result 1. CS t, x) + γv x t S t, x) is a concave function of R t, x). Structural Result 2. The optimal ecision x t is positive an finite an Structural Result 3. x CS t, x t ) + γ x V x t S t, x t ) = 0. 13) V t S t ) = ρ E µ p + 1 κ τ) p t µ p )] 14) R t R t+1 +γe ρ E x + R ) ] t+1 V t+1 S t+1 ) S t, x t. R t R t+1

16 16 Article submitte to Operations Research; manuscript no. OPRE Structural Result 4. V t S t ) is a concave function of R t. Structural Result 5. ρ E µ p + 1 κ τ) p t µ p )] V t S t ) 1 µ p + 1 κ τ) p t µ p )]. 15) R t See 2 in the e-companion to this paper for the proof of the above results. In Structural Result 3, which shows the recursive relationship between the marginal value functions, the meaning of the term ρ E µ p + 1 κ τ) p t µ p )] is obvious; if we ha an extra R t amount of energy in storage, we can commit to sell it an gain R t ρ E µ p + 1 κ τ) p t µ p )] in expecte revenue. However, the secon term requires some analysis. From 2), we know that R t+1 ρ E x + R { t+1 1 ρe ρ x=x R t = R, if x t < Y t+1, R t + Y t+1 x t ) < R max. t 0, otherwise. escribes the conversion loss that occurs when we use the energy that is put into the storage when we generate more electricity than we nee to satisfy the commitment. Therefore, the term R t+1 E ρ E x + R ) ] t+1 V t+1 S t+1 ) S t, x t R t R t+1 can be seen as the expecte portion of the marginal future value function that is save by not having to go through the process of energy conversion. 4. Main Result - Infinite Horizon Analysis In this section, we erive the marginal value function an the corresponing optimal policy for avance energy commitment that maximizes the expecte revenue in the infinite horizon case. However, while we can obtain the value of always having a storage as shown in this paper, it is important to note that the cost of always having storage is not the cost of installing the storage

17 Article submitte to Operations Research; manuscript no. OPRE once in the beginning. Batteries have finite lifetime, an we might have to re-install them every ten years, for example. We let T an rop the inex t from the value function: Then, V S t ) satisfies T ] V S t ) = lim E γ t t CS t, X π S t )) S t. T t =t V S t ) = max x R + {CS t, x) + γe V S t+1 ) S t, x]} = CS t, x t ) + γv x S t, x t ). Since the structural properties shown in the previous section hols true for all T, V S t ) maintains those structural properties. In 4.1, we erive the optimal policy using the main assumptions state in 2 an the structural results shown in 3. We first state: Theorem 1. The optimal policy, when the electricity generate from the win farm is uniformly istribute from θ t to θ t + β, is given by x t = X π S t ) = ρ E R t + θ t + µ pk 1 + p t µ p ) 1 κ τ) K 2 m µ p + p t µ p ) 1 κ τ)] + b β 16) where an K 1 = 1 γ ρ E 1 ρ E ρ E K 2 = 1 γ 1 κ τ) 1 ρ E exp γ 1 ρ E ) 1 β ] ) R max 1, exp γ 1 κ τ) 1 ρ E ) 1 β ] ) R max 1. Before proving 16), we first analyze its components. Since ρ E R t is the amount of electricity that can be prouce by converting the energy in storage an θ t is the amount of electricity that is certain to be prouce, ρ E R t + θ t can be seen as the riskless term. Since there is a limit on the size of the storage an we lose energy if the storage is full, we always want to commit to sell at least ρ E R t + θ t. The issue is then how much more to commit relative to this base level. Over-commitment is costly because the expecte penalty always excees the expecte spot price. Uner-commitment is costly for two reasons. First, excess prouction must be store an storage

18 18 Article submitte to Operations Research; manuscript no. OPRE is not free since the roun-trip efficiency is less than 1. Secon, since there is a limit on the amount of energy you can store, R max, if we commit too little an prouce too much we lose the prouction that cannot be store. So the optimal extra commitment over the base level must balance the cost of over-commitment an uner-commitment. β is the uncertainty in the electricity prouction, an committing µ p K 1 + p t µ p ) 1 κ τ) K 2 m µ p + p t µ p ) 1 κ τ)] + b 17) fraction of β achieves the balance between the cost of over-commitment an the cost of unercommitment. Note that the solution to a typical newsvenor problem states that the venor shoul always try to satisfy a fixe fraction of the ranom eman Khouja 1999), Petruzzi an Daa 1999)). However, in our case, the fraction is a function of the price because we can speculate on the movement of the price that is mean-reverting. In 4.2, we obtain the stationary istribution of the storage level corresponing to the optimal policy. In 4.3, we erive the economic value of the storage as the relative increase in average revenue ue to the existence of the storage Optimal Policy In this section, we prove the optimal policy 16) by first eriving the marginal value function. From Structural Result 2, we know that the optimal ecision x t must satisfy x CS t, x t ) + x V x S t, x t ) = ] x CS t, x Rt+1 t ) + γe V S t+1 ) S t, x t x R t+1 = 0. Therefore, in orer to compute x t, we only nee to know the erivative of V S t+1 ) with respect to R t+1, an we o not nee to know V S t+1 ) itself. To erive lemma: R t+1 V S t+1 ), we nee the following Lemma 1. x t + R max R t θ t + β, t. 18)

19 Article submitte to Operations Research; manuscript no. OPRE Proof: See 3 in the e-companion to this paper. The proof utilizes the inequality 10). We know that θ t + β x t is the maximum amount of excess electricity that can be left over after fulfilling the commitment. Suppose that the inequality 18) oes not hol. Then, θ t + β x t ) R max R t, inicating that there is always enough room left in the storage to accommoate all of the excess electricity, implying that there is no risk of uner-commitment at all. However, we have restricte the size of the storage as shown in 10) precisely to avoi such a situation. We know that the optimal policy ought to balance the risk of uner-commitment an the risk of overcommitment. The above lemma allows us to compute R t V S t ) from which we can erive the optimal policy. We first state: Theorem 2. V S t ) = ρ E µ p exp γ 1 ρ E ) 1 R t β +ρ E p t µ p ) 1 κ τ) exp )] Rmax R t γ 1 κ t) 1 ρ E ) 1 β Rmax R t )]. 19) Proof : Here, we show a conense version of the proof by omitting various algebraic steps. See 4 in the e-companion to this paper for a etaile proof. We prove the theorem by using backwar inuction in the finite horizon setting an letting T go to infinity. First, we make the inuction hypothesis that R T i V T i S T i ) = ρ E µ p +ρ E p T i µ p ) 1 κ τ) for some i 0, an prove that i j=0 i j=0 1 γ 1 ρ E ) 1 j! β 1 j! )] j Rmax R T i 20) γ 1 κ t) 1 ρ E ) 1 β )] Rmax R T i i+1 1 V T i+1) S T i+1) ) = ρ E µ p γ 1 ρ E ) 1 )] j Rmax R T i R T i+1) j! β j=0 ) i+1 1 +ρ E pt i+1) µ p 1 κ τ) γ 1 κ t) 1 ρ E ) 1 )] j Rmax R T i+1). j! β From 11), we know that j=0 R T V T S T ) = ρ E µ p + ρ E p T µ p ) 1 κ τ).

20 20 Article submitte to Operations Research; manuscript no. OPRE Therefore, the expression for R T i V T i S T i ) shown above is true for i = 0. From 2), we can show that = R t+1 ρ E x + R t+1 R t { 1 ρ E ) 1 j! ) 1 j! R max R t Rmax R t+1 Y t+1 x t ) 0, otherwise. ) j x=x t ] j, if x t < Y t+1, R t + Y t+1 x t ) < R max.. Next, by 18), Then, we can show E R t+1 ρ E x + R ) t+1 1 R t j! From Structural Result 3, R T i+1) V T i+1) S T i+1) ) f t y) = 1 β, x t y x t + R max R t. Rmax R t+1 ) j S t, x t = ρ E µp + 1 κ τ) p T i+1) µ p )) + γe ρ E R T i i+1 = ρ E µ p j=0 1 γ 1 ρ E ) 1 j! β ) i+1 +ρ E pt i+1) µ p 1 κ τ) ] = 1 ρ E ) 1 β x + R T i )] j Rmax R T i+1) j=0 1 j! R T i+1) γ 1 κ τ) 1 ρ E ) 1 β Therefore, 20) is true for i 0. Next, substitute t for T i + 1). Then, T t 1 V t S t ) = ρ E µ p γ 1 ρ E ) 1 R t j! β j=0 T t +ρ E p t µ p ) 1 κ τ) t T. If we let T go to infinity, R t V S t ) = lim T = ρ E µ p exp V t S t ) R t j=0 γ 1 ρ E ) 1 β +ρ E p t µ p ) 1 κ τ) exp )] j Rmax R t 1 γ 1 κ τ) 1 ρ E ) 1 j! β )] Rmax R t γ 1 κ τ) 1 ρ E ) 1 β 1 j + 1)! ) j+1 Rmax R t. ) ] V T i S T i ) S T i+1), x T i+1) R T i )] j Rmax R T i+1). )] j Rmax R T i+1), Rmax R t )], t.

21 Article submitte to Operations Research; manuscript no. OPRE To compute the optimal ecision x t at time t, all we nee to know is R t+1 V S t+1 ). Since we now know what Proof of 16): From 19), R t+1 V S t+1 ) is, we are reay to prove 16). Rt+1 E x = µ p ρ E 1 ρ E ] V S t+1 ) S t, x R t+1 exp γ 1 ρ E ) 1 β p t µ p ) 1 κ τ) 2 ρ E 1 ρ E exp R max ] ) 1 γ 1 κ τ) 1 ρ E ) 1 β ] ) R max 1, 21) x ρ E R t + θ t. To see the erivation of 21), see 5 in the e-companion to this paper. Then, from Structural Result 2, we know that the optimal ecision x t must satisfy x CS t, x t ) + γ x V x S t, x t ) = p t,t+1 mp t,t+1 + b) F t x t ρ E R t ) + γe = 0, Rt+1 x ] V S t+1 ) S t, x t R t+1 where p t,t+1 := E p t+1 F t ] = µ p + 1 κ τ) p t µ p ). Therefore, mp t,t+1 + b) 1 β x t ρ E R t θ t ) ] Rt+1 = p t,t+1 + γe V S t+1 ) S t, x t x R t+1 ρ E = µ p + p t µ p ) 1 κ τ) γµ p exp 1 ρ E γ p t µ p ) 1 κ τ) 2 ρ E exp 1 ρ E = µ p K 1 + p t µ p ) 1 κ τ) K 2. γ 1 ρ E ) 1 β R max γ 1 κ τ) 1 ρ E ) 1 β ] R max ) 1 ] 1 ) Then, x t = ρ E R t + θ t + µ pk 1 + p t µ p ) 1 κ τ) K 2 m µ p + p t µ p ) 1 κ τ)] + b β.

22 22 Article submitte to Operations Research; manuscript no. OPRE Note that both K 1 an K 2 increases when ρ E, R max, or γ is reuce. We know that the optimal amount shoul naturally epen on the penalty, the roun-trip efficiency, the maximum storage limit, an the iscount factor as follows. First, it shoul ecrease with increasing penalty, as being short incurs the penalty. Secon, it shoul increase with reuce roun-trip efficiency, as being long implies paying to store losing energy). Thir, it shoul ecrease with increasing maximum storage. If our storage capacity is greater, we lose less of the energy we o not sell, an we can affor to be more conservative an commit less. Fourth, it shoul increase with ecreasing iscount factor, because the value of what we store now to use in the future ecreases with the iscount factor. Next, suppose storage evices with sufficient capacities become ubiquitous in the future an hence electricity becomes a very liqui asset just like stocks. Then, arbitraguers taking avantage of preictable patterns such as mean-reversion will cause the electricity prices to behave more an more like a martingale. Corollary 1. If κ = 0, implying that p t ) t 0 is a martingale, then K 1 = K 2 an x t = ρ E R t + θ t + p t mp t + b K 1β. 22) If the price is a martingale, it is stochastically constant an we cannot speculate on the future movement of the price. Then, the fraction is just irectly proportional to the ratio between the current expecte spot market price an the penalty price Stationary Distribution of the Storage Level Now that we have the optimal policy 16), we want to assess the expecte value of storage corresponing to the policy. In orer to obtain a close-form expression for the expecte value of storage, we must analyze the ynamics of our system at the steay-state an erive the stationary istribution of the storage level. Denote Z t := µ pk 1 + p t µ p ) 1 κ τ) K 2 m µ p + p t µ p ) 1 κ τ)] + b, t.

23 Article submitte to Operations Research; manuscript no. OPRE From 2), we know that R t+1 is a function of R t, x t, Y t+1 ). Since Y t+1 is a function of θ t an x t is a function of R t, Z t, θ t ) as shown in 16), we can think of R t+1 as a function of R t, Z t, θ t ). However, because θ t is the amount of electricity that we are certain to prouce an commit, we know that R t+1 in fact oes not epen on θ t. Thus, R t+1 is a function of R t, Z t ). Therefore, if the ranom process Z t ) t 0 is stationary ergoic, the process R t ) t 0 will reach a steay-state. Since Z t ) t 1 is riven by p t ) t 1, we first nee to know the istribution of p t ) t 1 in steay state. Here we use the term steay-state to refer to the unconitional process. Proposition 1. At steay-state, p t N Proof: We know that 23) is true if an only if 23) implies Suppose 23) is true. Then, p t+1 N 1 κ τ) p t µ p ) N ) σp 2 µ p,. 23) 1 1 κ τ) 2 ) σp 2 µ p,. 1 1 κ τ) 2 Since p t+1 is inepenent from p t an p t+1 N 0, σ 2 p), 1 κ τ) p t µ p ) + p t+1 N 0, 1 κ τ)2 σ 2 p 1 1 κ τ) 2 ) ) σp 2 0,. 1 1 κ τ) 2. Then, p t+1 = µ p + 1 κ τ) p t µ p ) + p t+1 N ) σp 2 µ p, 1 1 κ τ) 2 Since Z t is a eterministic function of p t, Z t ) t 1 reaches steay-state when p t ) t 1 reaches steay-state. Although real-time electricity spot prices can be negative ue to tax subsiies, it will be extremely rare for the ay-ahea forecast price to be negative. Thus, in practice, we may assume that the price is always going to be positive. The first an secon moments of Z t at steay-state given p t 0 is ] µp K 1 + ε µ p ) 1 κ τ) K 2 Z 1 := E m µ p + ε µ p ) 1 κ τ)] + b ε 0, 24)

24 24 Article submitte to Operations Research; manuscript no. OPRE an where Also, efine an Z 2 := E ) 2 µp K 1 + ε µ p ) 1 κ τ) K 2 ε 0] m µ p + ε µ p ) 1 κ τ)] + b ε N ) σp 2 µ p,. 1 1 κ τ) 2 ] µ p + ε µ p ) 1 κ τ) Z 1 := E m µ p + ε µ p ) 1 κ τ)] + b ε 0, Z 2 := E ) 2 µ p + ε µ p ) 1 κ τ) ε 0], m µ p + ε µ p ) 1 κ τ)] + b 25) corresponing to the case where R max = 0, which makes K 1 = K 2 = 1. Z 1,Z 2, Z 1, an Z 2 can be easily compute via Monte-Carlo simulation using sample realizations of ε greater than zero. Proposition 2. Then, the stationary istribution of R t corresponing to the steay-state is f Rt r) = r P R t r] = Z 1 δ r) + Z 1 + ρ ) ] Rρ E 1 ρr ρ E ) 1 ρe ) exp r 1 {0 r Rmax} 1 ρ E β β ρ E + Z 1 1 ρ E ) ) ]) 1 ρe ) exp R max δ r R max ), 26) 1 ρ E β where δ ) enotes the Dirac-elta function. Proof: Here, we show a conense version of the proof by omitting various algebraic steps. See 6 in the e-companion to this paper for a etaile proof. From 2) an 16), we can show that P R t+1 = 0 R t ] = P R t+1 = 0 ] = Z 1 an P R t+1 = R max R t ] = 1 Z 1 R max β + 1 ρ E ) R t, β in the steay-state. Also, from 2), we can show that { ρe if 0 < u < R β t f Rt+1 R t u R t ) = 1 if R. β t u < R max

25 Article submitte to Operations Research; manuscript no. OPRE Therefore, we can write the conitional probability ensity function as f Rt+1 R t u R t = r) = Z 1 δ u) + ρ E β 1 {0 u<r} + 1 β 1 {r u R max} + 1 Z 1 R max β + 1 ρ ) E ) r δ u R max ), β where δ ) enotes the Dirac-elta function. Since P R t = 0] = Z 1 in the steay-state, we know that the stationary istribution can be written as R max f Rt r) = Z 1 δ r) + g r) 1 {0 r Rmax} + 1 Z 1 g r) r δ r R max ), for some function g r). By efinition, the stationary istribution must satisfy f Rt+1 u) = R max r=0 f Rt+1,R t u, r) r = R max r=0 r=0 f Rt+1 R t u R t = r) f Rt r) r = f Rt u). By computing the integral an matching the terms, we can show that gu) = Z 1 1 ρ E ) β + ρ E β + 1 ρ E ) β Taking the erivative with respect to u on both sie gives Then, we can show that gr) = Z 1 + g u) = 1 ρ E ) gu). β ρ E 1 ρ E ) 1 ρr ρ E ) β u r=0 g r) r. ] 1 ρe ) exp r β an 1 Z 1 R max r=0 1 g r) r = 1 ρ E 1 ρ E + Z 1 1 ρ E ) ) ]) 1 ρe ) exp R max. β The stationary istribution 26) shows that if the roun-trip efficiency is lower, the probability of hitting the capacity limit R max is lower while the probability of epleting the storage is higher, as expecte.

26 26 Article submitte to Operations Research; manuscript no. OPRE Economic Value of the Storage Using the stationary istribution of the storage level obtaine in the previous section, we can compute the following: Corollary 2. In steay-state, E R t ] = R max Z ρ E ρ E 1 ρ E ) ρr β 1 ρ E ] ) 1 ρr ρ E ) exp R max 1 β 27) an the expecte revenue in steay-state is C Rmax SS : = E C S t, x t )] = µ p ρ E E R t ] + µ p E θ t ] + µ p Z 1 β mµ p + b) β 2 Z 2 = µ pρ E R max µ p β Z 1 + ρ ) ] ) Rρ E ρe 1 ρr ρ E ) exp R max 1 1 ρ E 1 ρ E 1 ρ E β +µ p β Z 1 m Z ) + µ p µ Y bβ Z ) See 7 in the e-companion to this paper for the erivation of 27) an 28). We know that K 1 = K 2 = 1 if R max = 0. Therefore, from 28), if we o not have a storage an R max = R t = 0, t, the expecte revenue at steay-state woul be is C 0 SS := µ p β Z 1 m Z ) + µ p µ Y bβ Z 2 2. Then, the relative increase in the expecte revenue in steay-state ue to the existence of storage ψ : = CRmax SS CSS 0 CSS { 0 ρe R max = 1 ρ E β + Z 1 Z ) Z 1 + ρ E ) ρe 1 ρ E m + b ) Z 2 µ Z ) } 2 / p 1 ρ { E exp 1 ρ E ) R ] max β Z 1 m Z µ Y β b Z2 µ p 2 ) 1 }. 29) 5. Numerical Results In the previous section, we have erive the optimal commitment policy an the corresponing value of the storage assuming that the forecast of electricity generate from the win farm is uniformly istribute. However, the hourly win spee ata obtaine from the North American Lan

27 Article submitte to Operations Research; manuscript no. OPRE W W W W W W N N N N/A N N/A Table 1 Mean of the cube of the spee of the win in January 2000 Data Assimilation System NLDAS) project shows that when forecasting the cube of the spee of the win, a truncate Gaussian istribution fits the ata better than a uniform istribution. In this section, we simulate the win energy process using a truncate Gaussian istribution an compare the relative increase in revenue ue to the existence of storage compute numerically by implementing our policy 16) to the one compute theoretically from the equation 29). From the NLDAS project, we extracte win spee ata from 22 locations across the Unite States. Since the win characteristics vary throughout the year ue to seasonal effects, it is common to assume that the win process is time-invariant over a one month perio but not beyon that Ettoumi, et al. 2003)). Therefore, we use separate moel parameters an corresponing policies for each month. We foun that the thir-orer correlation is very small compare to the first an secon orer correlation, an represent Y t ) t 1, the energy generate from our win farm, as a secon-orer AR process: Y t+1 = µ Y + α 0 Y t µ Y ) + α 1 Y t 1 µ Y ) + ŷ t+1, 30) for some µ Y, α 0 an α 1. When we implement our policy 16), we assume ŷ t ) t 1 is i.i. with istribution U β 2, β 2 ), for some β. β is compute by matching β 2 12 to the variance of the resiual in the AR process, ŷ t ) t>1. µ Y s in m 3 /s 3 ) for the selecte 22 locations compute using the January 2000 ata, for example, are given in Table 1 an β s in m 3 /s 3 ) are given in Table 2. From Table 1 an Table 2, we can see that µ Y s an β s are comparable in magnitue, implying that win energy prouction is highly volatile. After we compute µ Y, α 0,α 1 an β using the NLDAS ata, we generate win energy processes ) Y t ) t 1 from equation 30) where ŷ t ) t>1 is i.i. an ŷ t N, t. However, when we are 0, β2 12 computing our commitment from our policy 16), we assume ŷ t U β 2, β 2 ). From 6),

28 28 Article submitte to Operations Research; manuscript no. OPRE W W W W W W N N N N/A N N/A Table 2 Sprea of the cube of the spee of the win in January W W W W W W N N N N/A N N/A Table 3 Relative increase in revenue compute by implementing our policy 16) θ t = µ Y + α 0 Y t µ Y ) + α 1 Y t 1 µ Y ) β 2. Next, we fit the hourly spot market price provie by a utility company to the process p t+1 = µ p + 1 κ τ) p t µ p ) + p t+1, 31) where an p t ) t 0 are i.i. with istribution N 0, σ 2 p). In our experiments, we use ρe = 0.75, γ = 0.99, µ p = 49.9, σ p = 47.46, τ = 1, κ = , m = 1.6, b = 67.5, an Rmax β Z 1 =.1912, Z 2 =.0467, Z 1 =.3270, an Z 2 = This gives = 0.5. Then, ) µy ψ = / β ) We implemente our policy 16) 100 times by generating the prices from 31) an the win energy process from 30) using the coefficients µ, α 0,α 1 an β. Then, we compute the relative increase in revenue ue to the existence of storage for each implementation of our policy an foun the average of those values over the 100 experiments. Next, we compute the relative increase in revenue irectly from equation 29). The relative increase in revenue compute by implementing our policy 16), average over 36 months, is given in Table 3. The relative increase in revenue compute from 29) an hence 32), is given in Table 4. From the above tables, we can see that the relative increase in revenue obtaine through a sample run implementing our policy 16) is comparable to the relative increase in revenue compute