The Valuation of Natural Gas Storage: A Knowledge Gradient Approach with. Non-Parametric Estimation

Transcription

1 The Valuation of Natural Gas Storage: A Knowledge Gradient Approach with Non-Parametric Estimation By: Jennifer Schoppe Advisor: Professor Warren B. Powell Submitted in partial fulfillment Of the requirements for the degree of Bachelor of Science in Engineering Department of Operations Research and Financial Engineering Princeton University June 2010

2 I hereby declare that I am the sole author of this thesis. I authorize Princeton University to lend this thesis to other institutions or individuals for the purpose of scholarly research. Jennifer Schoppe I further authorize Princeton University to reproduce this thesis by photocopying or by other means, in total or in part, at the request of other institutions or individuals for the purpose of scholarly research. Jennifer Schoppe ii

3 Dedication To my unforgettable Princeton experience, my parents who gave me the opportunity to take the journey, and my sister who was with me through every moment. iii

4 Acknowledgements There are many people who helped bring this thesis into existence, and I would like to extend my gratitude though the thanks they deserve extends beyond what can be expressed on this page. Within the ORFE Department, I would first like to thank my advisor Professor Warren Powell for his enthusiasm, insight, and encouragement to make my thesis better. My gratitude extends to Dr. Hugo Simao for his computational expertise and Michael Coulon for his knowledge on energy pricing processes. A special thanks goes to Emre Barut, for allowing this thesis the opportunity to be one of the first to test his research in non-parametric estimators for knowledge gradient methods. His time and energy spent in providing both computational and theoretical support is greatly appreciated. I would like to thank James Schoppe at Iberdrola Renewables and Richard Adamczyk at Enstor Operating Company for providing me with invaluable resources regarding the natural gas industry and storage. I would like to acknowledge Doug Eshleman and Mike Hasling for their time and patience in answering my questions as I navigated through the world of computer programming. I would also like to thank Stephanie Schoppe for copyediting this work. I have been blessed to have many people in my life who have offered me love and encouragement through all my achievements including the creation of this thesis. They are my parents, my grandparents, my sister, and my friends. Many thanks to all. iv

5 Abstract This thesis examines the problem of natural gas storage valuation focusing on the valuation of high-deliverability gas storage facilities, specifically salt cavern storage. We present capable valuation analysis and optimization methodologies that accurately account for the various operating characteristics of real storage facilities as well as the complex stochastic nature of natural gas prices. We use an extension of the Knowledge Gradient Algorithm, a sequential learning method, which incorporates non-parametric estimation in order to present a method that produces a quality valuation with minimal computational burden. The results of the method will present an optimal strategy for the operation of the natural gas storage facility, as well as yield a value for the facility. v

6 Table of Contents CHAPTER 1 Introduction Natural Gas: From Wellhead to Market Exploration and Extraction Production and Transport Storage Distribution and Marketing Natural Gas Markets... 9 Natural Gas Spot Market...10 Natural Gas Futures Market Traditional Use of Underground Natural Gas Storage Seasonal Cycling Peaking Services Speculative Market Services Types of Underground Natural Gas Storage Depleted Reservoirs Aquifers Salt Caverns Storage Contract Overview CHAPTER 2 Literature Review Spread Option Optimization Approach Approximate Dynamic Programming (ADP) Approach CHAPTER 3 Knowledge Gradient Methods Knowledge Gradient with Independent Measurements Knowledge Gradient for Correlated Beliefs Knowledge Gradient with Non-Parametric Estimation Kernel Estimation KGNP Procedure and Derivation...41 CHAPTER 4 Pricing Models...44 vi

7 4.1 Geometric Brownian Motion Spot Price Process Strength of Mean Reversion (α)...48 Exponential Smoothing...49 Volatility...50 Simulation Results Forward Price Process CHAPTER 5 Model and Policies Modeling Assumptions Problem Set-Up and Terminology Parameters Decision Variables and Constraints Exogenous Variables System State Transition Functions Contribution Function Objective Function Policies Swing Trading Policy 0 Short-Term Speculation Policy 1 Long Trend Speculation Policy 2 Long and Short Trend Speculation Policy 3 Forward Hedging...72 CHAPTER 6 Results Model and Algorithm Set-Up Policy 0: Results for Short-Trend Speculation Policy 1: Results for Long Trend Speculation Policy 2 Long and Short Trend Policy 3 Forward Hedge Policy Analysis vii

8 6.6.1 Withdrawal Rate Adjustment Volatility Fluctuation KGNP Efficiency Real World Testing and Comparison...90 CHAPTER 7 Conclusion and Further Research...93 Appendix: Code...96 viii

9 Table of Figures Figure 1- Natural Gas Powered Generation Plant... 2 Figure 2 - Salt Cavern Facilities... 3 Figure 3 - Preparing Hole for the Explosive Charges Used in Seismic Exploration... 5 Figure 4 - Shale Deposits... 6 Figure 5 - Alaskan Pipeline Carrying Natural Gas... 7 Figure 6 - Natural Gas from Wellhead to End-Users... 8 Figure 7 - Henry Hub in Erath, Louisiana Figure 8 - Fluctuations in Underground Storage Volume Figure 9 - Breakdown of Storage Portfolio in United States Figure 10 - Depleted Reservoir Figure 11- Aquifer Storage Figure 12 - Salt Cavern Figure 13 - Illustration of Knowledge Gradient Figure 14 Knowledge Gradient with Correlated Beliefs at Measurements Figure 15- Knowledge Gradient with Correlated Beliefs at Measurement Figure 16 - Knowledge Gradient with Correlated Beliefs Estimated Surface Figure 17 - Plot of Kernel Functions K(u) Figure 18 - Henry Hub Historical Natural Gas Spot Prices Figure 19 - Natural Gas Spot Prices with Exponential Smoothing, Beta = Figure 20 - Sample Paths for 1-Year of Simulated Natural Gas Spot Prices Figure 21 - Simulated Month Ahead Forward Prices for One Year Figure 22 - Month Ahead Forward Prices with Spot Prices for One Month Figure 23 Swing Trend with 10% Filtering Figure 24 - Swing Trend with 5% Filtering Figure 25 - Policy 0: Second Measurement Figure 26 - Policy 0: 25 Measurements Figure 27- Policy 0: 100 Measurements Figure 28 - Policy 0: 500 Measurements Figure 29 - Policy 1: First Measurement ix

10 Figure 30 - Policy 1: 100 Measurements Figure 31 - Policy 1: 500 Measurements Figure 32 - Withdrawal Rate's Effect on Storage Value Figure 33- Policy 0, KGNP Result for Doubled Volatility Figure 34 - Policy 0: S-Curve Figure 35 - Policy 1: S-Curve Figure 36 - Price Paths for 2005 and 2009 Natural Gas Spot Prices x

11 CHAPTER 1 Introduction In late November 2009, the burgeoning glut of the U.S. natural gas supply ventured into uncharted terrain as the Energy Information Administration (EIA) reported the volume of working gas within U.S. storage to have hit trillion cubic feet (Tcf). This expansion is mirrored in the demand for natural gas, which has called more and more supply away from its historical use in residential and industry sectors to the power generation sector. Natural gas is playing an increasing role in power generation, and in 2009, 23,475 MW of new generation capacity was planned in the U.S. with over 50% being natural gas fired additions (Energy Information Administration 2004). Figure 1 shows one of the many rapidly emerging gas powered generation plants. The natural gas market is rapidly changing, and natural gas storage will play a significant role as it buffers demand and regulates supply. However, with shifting supply and demand, other functions of storage can be utilized such as with salt caverns, which exhibit high deliverability, high injection and withdrawal capacity,

12 CHAPTER 1: Introduction that can quickly respond to changing gas prices; and thus are utilized as an arbitrage mechanism. Storage owners and service providers will need to reexamine the value of their facilities which is derived Figure 1- Natural Gas Powered Generation Plant from the expected profit from the operations of a facility. The problem of the valuation of a natural gas storage facility is not new, and literature has been produced within the industry as well as in academic circles. Industry professionals employ linear optimization with Monte Carlo methods (Byers 2006) while the academic community supports the use of stochastic optimization with an Approximate Dynamic Programming approach (Lai, Margot and Secomandi 2008). Unfortunately, there remains advantages and disadvantage of both techniques rendering it ambiguous to decipher the superior procedure. We will examine the problem of natural gas storage valuation focusing on the valuation of high-deliverability gas storage facilities, specifically salt cavern storage as pictured in Figure 2. We will strive to present capable valuation analysis and optimization methodologies that accurately account for the various operating characteristics of real storage facilities as well as the complex stochastic nature of natural gas prices. 2

13 CHAPTER 1: Introduction Figure 2 - Salt Cavern Facilities (Energy Information Administration 2004) To solve the problem we will utilize stochastic optimization methods within the realm of Optimal Learning. The idea encompasses a stochastic search over policies which dictate the operational decisions of the storage facility. We assume the policies are composed of a set of tunable parameters, and through sequential measurements of the alternatives, estimates of each policy s performance can be ascertained. The goal is to choose the alternatives, and subsequently the policy, 3

14 CHAPTER 1: Introduction which will produce the best performance. A strong performer within sequential learning algorithms is the knowledge gradient which was developed by Gupta & Miescke (1996) and later analyzed by Frazier et al. (2008). This algorithm strives to maximize the amount of knowledge obtained with each measurement in an effort to obtain an improved solution with the new knowledge. We will use an innovative method newly added to the literature on the knowledge gradient algorithm that evaluate problems with correlated beliefs using non-parametric estimation (Barut and Powell 2010). The results of the method will account for an optimal strategy for the operation of the natural gas storage facility, as well as yield a value for the facility. 1.1 Natural Gas: From Wellhead to Market Though it may be simple to flip on one s burner in the kitchen, the process of getting natural gas from the ground and into everyday life is actually an extensive and complex process. This section provides an overview of the process, journeying from exploration to marketing of the natural gas that will be sold for homes and industry Exploration and Extraction The process begins with geologists and geophysicists, who use technology and knowledge of the properties of underground natural gas deposits to gather data and make educated guesses as to where natural gas deposits may exist thousands of feet below ground. The most important tool in exploration is the use of basic seismology which refers to the study of how energy moves through the Earth's crust 4

15 CHAPTER 1: Introduction and interacts differently with various types of underground formations. The energy can be mapped, providing an effective way to image both sources and structures Figure 3 - Preparing Hole for the Explosive Charges Used in Seismic Exploration (Energy Information Administration 2004) deep within the Earth. In both onshore and offshore exploration, seismic waves are created that can travel through the Earth's crust and generate the reflections needed to ascertain the presence of natural gas or the properties of a reservoir formation (NaturalGas.org 2004). Figure 3 shows operators preparing a hole for the explosive charges used in seismic exploration. After extensive measurements deem that the site may be enriched with a marketable quantity of natural gas, drilling can begin. However, the exploration team may be incorrect in its estimation of the existence of natural gas at a well site. If this occurs, the well is labeled a 'dry well,' and production does not proceed. Conversely, if the new well is drilled and comes in contact with natural gas deposits, it is developed to allow the extraction of the natural gas and is labeled as a 'productive' well (NaturalGas.org 2004). A common misconception is that all natural gas is extracted from natural gas wells. In fact, natural gas is extracted from a variety of sources oil wells, shale, and coalbeds. Oil wells which contain natural gas are known as associated wells. Even 5

16 CHAPTER 1: Introduction if the well is specifically drilled for oil extraction, natural gas will almost always be a byproduct of producing oil, since the small, light gas carbon chains come out of solution as it undergoes pressure reduction from the reservoir to the surface. An unconventional source of natural gas is shale, a fine-grained sedimentary rock which can contain natural gas within its Figure 4 - Shale Deposits layered formation pictured in Figure 4. In recent years, shale deposits have accounted for an increasing share of the natural gas production. As of November 2008, FERC estimated that there are 742 Tcf of recoverable shale gas in the United States. A second unconventional source is coalbed methane. Coal deposits are commonly found as seams that run underground. Many coal seams also contain natural gas, either within the seam itself or the surrounding rock which can be extracted and transported for production (NaturalGas.org 2004). Table 1 lists the sources of natural gas and the amount of natural gas extracted from each in the year Table 1 - Sources of Natural Gas and Production Volume (Energy Information Administration 2004) Type of Well Gross Production for 2008 % of Total Production Natural Gas Well 18,011,151 MMcf 64.84% Oil Well 5,844,798 MMcf 21.05% Shale Deposits 2,022,000 MMcf 7.28% Coalbed 1,898,400 MMcf 6.83% 6

17 CHAPTER 1: Introduction Production and Transport Although consisting of mostly methane, the gas extracted at the wellhead is considered raw and can be mixed with oil or other hydrocarbons such as ethane, propane, butanes, and pentanes. In order to maintain pipeline quality, regulations are in place to ensure only purified natural gas is transported. Typically, some of the needed processing is accomplished at the well site and later completed at processing plants connected through gathering pipelines. Some of the hydrocarbons associated with the natural gas are very valuable and are known as natural gas liquids (NGL). NGLs have a Figure 5 - Alaskan Pipeline Carrying Natural Gas variety of uses and can be sold separately from the natural gas (NaturalGas.org 2004). Once the natural gas has been purified, it enters into a complex network of pipelines that make up the transportation system for natural gas. The system is designed to quickly and efficiently transport natural gas from the plant, to areas of high natural gas demand for use. It is composed of interstate and intrastate pipelines which are responsible for transporting natural gas at high speeds and pressure throughout the country and within a particular state respectively. Natural gas will usually travel great distances before reaching the end user. Figure 5 shows the Alaskan pipeline which carries natural gas over 800 miles. 7

18 CHAPTER 1: Introduction Storage Should natural gas not be required at a certain time, it can be transported to storage facilities for when it is needed. Given our concentration on the valuation of natural gas storage facilities, great detail of the various types of facilities and the uses for storage will be discussed in Section 1.2 and Distribution and Marketing Distribution is the process of delivering natural gas to end users who receive the gas from local distribution companies (LDC). LDCs move small volumes of gas at low pressures over shorter distances to a great number of individual users. A visual review of the journey from wellhead to the individual users end-users can be found in Figure 6. To perform the task, an extensive network of distribution pipeline is present and is estimated to be over one million miles in the United States. LDCs typically offer bundled services that combine the cost of all upstream activities, including transportation and the purchasing price of the natural gas itself. This is offered in one price for the customers to pay. Figure 6 - Natural Gas from Wellhead to End-Users (Natural Gas Depot 2009) 8

19 CHAPTER 1: Introduction The increasingly important role of natural gas marketers is leading to innovative ways of supplying natural gas to small volume users. Natural gas marketing began in the 1980s after the deregulation of the natural gas commodity market and the open access policy of natural gas pipelines. Natural gas marketing is generally defined as the selling of natural gas, including selling services that a particular purchase requires; including transportation, storage, accounting, and any other service that is needed to facilitate the sale of natural gas. Marketers also participate in the purchase and subsequent resale of natural gas, thus it is not uncommon for the gas to have three to four separate owners before it actually reaches the end-user. In addition to the buying and selling of natural gas, marketers will use the financial markets and instruments to reduce their exposure to risk as well as earn money through speculating market movements Natural Gas Markets In the United States, natural gas is traded on both the spot and futures market. The price for natural gas is set by the New York Mercantile Exchange in the daily transactions of the commodity and greatly impacts the activities of marketers. Natural gas prices are also crucial to our problem of natural gas storage valuation where prices dictate the injection and withdrawal of natural gas from storage as speculators attempt to capitalize on market movements. 9

20 CHAPTER 1: Introduction Natural gas is priced and traded at different locations throughout the country at what is referred to as market hubs, which exist along the intersection of major pipeline systems. Although there are over Figure 7 - Henry Hub in Erath, Louisiana thirty major market hubs in the U.S., this paper will primarily concern itself with the Henry Hub located in Louisiana. Spot and future prices are set at Henry Hub and are seen to be the primary price set for the North American natural gas market. Natural Gas Spot Market The natural gas spot market is the daily market quoted in $/MMBtu, where natural gas is bought and sold for immediate delivery. The spot market would be the source to obtain the price of natural gas on a specific day. The spot market is traded every business day, and settled, paid and delivered on the next day. For example, if the transaction date was Wednesday, November 17, 2010 the settlement date would be Thursday, November 19, 2010 (CMEGroup 2010). Natural Gas Futures Market Natural gas is traded using forward contracts which are widely traded on the New York Mercantile Exchange (NYMEX). One natural gas contract has an energy value of 10,000 MMBtus. Natural gas forward contracts are Henry Hub contracts, meaning they reflect the price of natural gas for physical delivery at this hub. The prices of 72 forward contracts are available every business day for trade. A feature 10

21 CHAPTER 1: Introduction of the forward contracts is that they will expire on the third to last business day of every month. For example, the February 2010 contract will expire on January 27, When a contract expires and is settled, the buyer receives the gas every day for the specified month, receiving 1/30 of the contract every day (CMEGroup 2010). Both markets will be employed in testing our procedure to value natural gas storage. Chapter 4 will introduce modeling methods which will be utilized to simulate the price processes of the spot and future markets of natural gas. 1.2 Traditional Use of Underground Natural Gas Storage Storage facilities were developed to allow the production capacity of natural gas to be moved from one point in time to another. Natural gas that reaches its destination is not always needed right away, so it is injected into underground storage facilities where it can be stored for an indefinite period of time. This capability is utilized in Seasonal Cycling, where a facility stores gas to meet variations in seasonal load. Other uses for natural gas storage include Peaking Services, where facilities use gas capacity to respond to quickly changing conditions effecting demand of natural gas, and Speculative Market Services where high deliverability storage quickly responds to changing gas prices capitalizing on price movements at market centers Seasonal Cycling Production of natural gas produces a constant supply from the various sources mentioned in Section 1.1.2, contrary to the varying demand. Traditionally, the demand of natural gas is cyclical, with demand higher during the winter months 11

22 CHAPTER 1: Introduction due to the need to provide heat in residential and commercial locations. Corresponding to this seasonal pattern, natural gas will be injected into storage facilities throughout the non heating season, which typically runs from April to October. This is when production of natural gas exceeds demand. Subsequently, the excess natural gas stored during the summer is withdrawn during the heating season, which includes the months from November to March, when production falls short of the demand. Storage facilities involved in seasonal cycling are large and capable of holding enough natural gas to satisfy the long term seasonal demand requirement. Without natural gas storage, supply would be limited in satisfying increased demand during winter months. It is vital in its role of ensuring that excess supply delivered during summer months is available during the winter. Figure 8 depicts the seasonal pattern of natural gas storage withdrawal and injection. (NaturalGas.org 2004) Figure 8 - Fluctuations in Underground Storage Volume (Energy Information Administration 2004) 12

23 CHAPTER 1: Introduction Peaking Services Demand for gas during the winter can be highly volatile due to the uncertainty of weather patterns. An unexpected drop in temperature calls for the immediate delivery of natural gas from storage. This peak demand seen in winter is now being replicated during the summer as the new trend of natural gas fired electric generation, increases the demand for natural gas during the summer months due to the demand for electricity. Electricity demand is also dependent on the variability of weather as residential and commercial buildings power their air conditioners to combat the summer heat. Following these fluctuations of electricity load, the demand for natural gas oscillates from day to night and from weekday to weekends. Thus, not only is storage needed to store excess natural gas during lowpeak hours, but storage facilities must be able to provide high-deliverability of the natural gas to respond to the sudden short-term periods of high demand (NaturalGas.org 2004) Speculative Market Services After 1992, with the introduction of the Federal Energy Regulatory Commission s Order 636, interstate pipeline companies who owned all the natural gas flowing through their system including the gas held in storage were required to operate their storage facilities on an open access basis as part of the deregulation of the natural gas market. This has expanded the use of storage from its role as a backup supply source in times of excess demand to an agent of the financial markets. Now, marketers can move gas in and out of storage in order to capitalize on the changes in price levels as well as in combination with financial instruments such as 13

24 CHAPTER 1: Introduction futures and other option contracts. In order to adapt to its new role of profiting from market conditions, natural gas storage has focused on more flexible operations with high deliverability and rapid cycling in their inventories (Energy Information Administration 2004). 1.3 Types of Underground Natural Gas Storage There are three main types of underground storage: depleted gas and oil reservoirs, aquifers, and salt caverns. The breakdown in percent of each storage type that makes up the portfolio of U.S. storage is shown in Figure 9, and the specific characteristics of each are described in Sections However, all have similar operational characteristics. All underground storage has a capacity measured in Bcf (billion cubic feet), which can be divided amongst the amount of working gas and base gas within a facility. Often when a capacity of a storage facility is quoted, it is referring to the working gas capacity seeing as it is the amount which can be withdrawn and injected into a facility. Every storage facility comes attached with a maximum and minimum withdrawal and injection rates which are typically expressed in Bcf/day. The injection and withdrawal rates for natural gas fluctuate based on the amount of pressure (PSI) within the storage facility. Withdrawal rates share a direct relationship with pressure while injection rates maintain an indirect relationship. Pressure for a facility is also bounded by a maximum and minimum quantity which is determined by the volume, depth, and structure of a facility. These operational characteristics determine the operational flexibility of a facility. 14

25 CHAPTER 1: Introduction Figure 9 - Breakdown of Storage Portfolio in United States (Energy Information Administration 2004) Depleted Reservoirs The most prominent type of underground storage due to their wide scale availability is depleted gas or oil reservoirs. These storage facilities are gas or oil reservoir formations that have already been tapped of all their recoverable resource through earlier production, leaving an underground formation geologically capable of holding natural gas. As a result, storage facilities of this nature are abundant in producing regions (Dietert and Pursell 2000). Of the three types of underground storage, depleted reservoirs are the cheapest and easiest to develop, operate, and maintain. Using an already developed reservoir for storage presents the opportunity to reuse the extraction and distribution equipment left over from when the field was productive, reducing the cost of conversion to gas storage. However, the maturities of these reservoirs require a substantial amount of maintenance. 15

26 CHAPTER 1: Introduction In order to sustain pressure in depleted reservoirs, the facility maintains equal parts base and working gas. However, depleted reservoirs, having already been filled with natural gas, do not require the injection of what will become physically unrecoverable gas seeing as it already exists in the formation. Depleted reservoirs with high permeability and porosity are ideal for natural gas storage, porosity lending itself to the amount of natural gas it can hold and permeability determining the rate of flow of natural gas through the formation. Figure 10 - Depleted Reservoir This in turn determines the injection and withdrawal rate of working gas. Disadvantages of using depleted reservoirs are the uncertainty of capacity. The configuration of the geological formation is never fully used since it runs the risk that injected gas may diffuse into the outer veins of the formation and becomes inaccessible. Other disadvantages include a reservoir s limited cycling capabilities, where working gas volumes are usually cycled only once per season. In addition, reservoirs are characterized with having low deliverability and thus would not be well suited for peaking services. It is most typically employed for seasonal cycling (NaturalGas.org 2004). 16

27 CHAPTER 1: Introduction Aquifers Aquifers are underground permeable rock formations that act as natural water reservoirs. When reconditioned, these formations may be used as natural gas storage facilities where gas is injected on top of the water formation displacing the water further down within the structure. Most of these facilities are located in the upper Mid-West where there is a Figure 11- Aquifer Storage lack of depleted oil and gas reservoirs (Dietert and Pursell 2000). Advantages of aquifer storage include their close proximity to markets where other geological reservoirs are not readily available. Deliverability rates may also be enhanced due to the presence of an active water drive which increases the storage facilities overall pressure. The high deliverability allows the working gas volumes to cycle through the facility more than once per season. Aquifer storage is the least desirable form of storage due to its physical and economic disadvantages. A significant amount of time and money is spent testing the suitability of an aquifer for natural gas storage and subsequently developing the infrastructure needed for an effective natural gas storage facility. In addition, in aquifer formations, base gas requirements are as high as 80 percent of the total gas volume. Unlike base gas from depleted reservoirs, this base gas is unrecoverable in 17

28 CHAPTER 1: Introduction aquifer storage due to the risk of facility damage. This high base gas requirement increases the initial cost of capital for aquifer storage projects, thus limiting their number. Most aquifer storage facilities were developed when the price of natural gas was low, meaning this base gas was not very expensive to give up. However, with higher prices, aquifer formations are increasingly expensive to develop and most often other storage type projects will be contracted (NaturalGas.org 2004) Salt Caverns Salt cavern storage sites are high pressured, solution-mined cavities in existing salt dome caverns located at depths several hundred to several thousand feet below the earth s surface. They are accessible by one or more wells per cavern. Figure 12 depicts a typical salt cavern facility. Most of the salt cavern facilities are located in the high producing region of the Gulf Coast, with some bedded salt deposits located in the high consumption states of western Pennsylvania and New York (Dietert and Pursell 2000). Figure 12 - Salt Cavern Underground salt formations are well suited to natural gas storage allowing for little injected natural gas to escape from the formation unless specifically extracted. The walls of a salt cavern have the structural strength of steel making it resilient against degradation over 18

29 CHAPTER 1: Introduction the life of the facility. Bas gas requirements are the lowest of all three storage types, requiring on average only 33 percent of total gas capacity to the natural gas storage vessel which maintains very high deliverability rates, exceedingly higher than that of depleted reservoirs and aquifers. This allows for natural gas to be more readily withdrawn, sometimes on as little as an hour s notice, which is well suited for satisfying unexpected surges in demand. The caverns also offer operational flexibility having the ability to cycle working gas four to five times a year, reducing the per-unit cost of each thousand cubic feet of gas injected and withdrawn. This multiple cycling capability coupled with its high deliverability is why salt caverns are well suited for peaking services as well as responding to volatility in natural gas market prices for commodities traders. Drawbacks of this form of storage are volume limitations where each cavern size typically ranges from 5-10 Bcf of working gas, considerably smaller than capacity capabilities of depleted reservoirs and aquifers. In addition, start-up costs generated during cavern development are substantial, and the disposal of saturated salt water produced during the solution mining can be detrimental to the environment (NaturalGas.org 2004). A summary of the three different storage facilities and their defining characteristics and uses can be found in Table 2. 19

30 CHAPTER 1: Introduction Table 2- Storage Facility Characteristics (Eydeland and Wolyniec 2003) Facility Description Injection Withdrawal Operating Costs Major Use Depleted Fields Low deliverability, low cycling, high capacity Days Days High with some fuel losses Seasonal Cycling Salt Caverns High deliverability, high cycling, low capacity 20 Days 5-20 Days Low with minimal fuel losses Peaking Services Aquifers Low deliverability, low cycling, high capacity Days Days High with some fuel losses Seasonal Cycling 1.4 Storage Contract In order to use any type of storage facility above, a storage contract must be entered into. A natural gas storage contract will specify the term date for the party s use of the storage, the type of storage facility, as well as the physical constraints and operational costs of the facility. A specific catalog of these physical and operational components can be found in the example contract below. Table 3 - Example Storage Contract for Depleted Reservoir Storage (Enstor 2009) Term 2/1/2010-1/31/2012 Type Depth Maximum Working Gas Capacity Initial Working Gas Capacity Maximum/Minimum Injection Rate Maximum/Minimum Withdrawal Rate Gas Reservoir 7000 ft 30 Bcf 0 Bcf 400 MMcf/day 50 MMcf/day 1 Bcf/day 200 MMcf/day Fuel Injection Loss Spread 1.0%- 2.5% Maximum/Minimum Facility Pressure psi 20

31 CHAPTER 1: Introduction 1.5 Overview Now possessing a foundation in the natural gas industry, including the application of storage, we are able to advance towards our goal of valuing a storage facility in the form of a salt cavern. The analysis will begin in Chapter 2 with a study of linear optimization with Monte Carlo methods discussed by Byers (2006) as well as the Approximate Dynamic Programming approach introduced by Lai et al. (2008). In Chapter 3, we will describe Knowledge Gradient Algorithms in particular a knowledge gradient method which employs non-parametric estimation that will be utilized in solving the valuation problem. Chapter 4 presents the pricing process used to model the complex stochastic nature of natural gas spot and future prices. The simulations garnered in Chapter 4 will be utilized within the model outlined in Chapter 5, which describes the operational functions of a storage facility and will derive the costs and rewards achieved from its operation. The policies exercised in making the operational decisions for the storage facility will also be exhibited in Chapter 5. Chapter 6 will present the results of the knowledge gradient method, bestowing the policies of Chapter 5 with optimal parameters that maximize the operation of the storage facility, allowing for the deduction of the proper value for the salt cavern storage. Chapter 7 will impart any conclusions deduced from the results as well as extensions in the development of the methods used to solve the storage valuation problem. 21

32 CHAPTER 2 Literature Review Valuing a contract is difficult because it entails the dynamic optimization of inventory decisions with capacity and operational constraints while combating the uncertainty of natural gas pricing dynamics. Several approaches have been developed to tackle this problem: Forward Optimization optimizes the withdrawal and injection schedules given the current forward prices and storage constraints, Forward Dynamic Optimization (Spread Option Optimization) values the storage as a portfolio of spread options which is valued based on the forward curve, and Stochastic Dynamic Programming which depends on directly modeling the price processes and storage flows given the dynamics (Eydeland and Wolyniec 2003). This review will encompass the two latter approaches given that they are favored by natural gas industry professionals and academic scholars respectively.

33 CHAPTER 2: Literature Review 2.1 Spread Option Optimization Approach Within the natural gas industry, there is a preference of modeling the full dynamics of the futures term structure using high dimensional forward models. Though this high dimensionality may hinder a stochastic dynamic programming approach, practitioners have found satisfaction in the solution rendered by spread option valuation methods combined with a linear program (LP) which is embedded within a Monte Carlo simulation. Byers (2006) depicts this process as a two stage valuation problem. In the first stage, linear optimization is used to determine spot market transactions, which is the actual purchase and sale of the physical gas. The linear optimization model is a profit maximization problem, where profit is defined by N Profit = t=1(p bid t w t + P ask t i t c i t c w t ) δ. Parameters include the withdrawal and injection volume, w t and i t respectively, the number of contract periods within the tenure of the facility, N, the cost of withdrawal and injection, c t w and c t i, and the price variables calculated from the forward price curve and bid-ask spread curve, P t bid and P t bid. As with most optimization problems, the maximum profit obtained through the trafficking of the commodity is subject to constraints. Physical and operational constraints are put in place to ensure that the purchase and sale of natural gas does not violate the withdrawal and injection capacities of the storage facility during a specified period. These capacities depend on the maximum capacity of the facility, 23

34 CHAPTER 2: Literature Review the current inventory, and the future commitment to deliver or accept the delivery of expiring natural gas futures contracts. This profit maximization problem given as an LP is actually solved many times during the second stage of the valuation which iterates over the first stage calculating the marginal contributions of changes in the storage portfolio from day to day using Monte Carlo methods. The result of this technique is an efficient and easily obtained value for natural gas storage assets. This technique is extremely attractive to industry professionals due to the simplicity in the linear structure of the model which can be solved utilizing simple optimization methods such as the simplex method. Software for solving linear optimization problems is widely available and can handle substantial sized models. However, the linear optimization structure of the model which allows for efficient computation is also the root of debate regarding the valuation results. Computing the natural gas storage valuation as a linear optimization problem assumes that the storage facility operations are deterministic. In fact, natural gas storage decisions follow a non-deterministic process meaning that the system's state is determined both by the process's predictable actions and by random elements. Natural gas storage decisions heavily depend on the stochastic processes exhibited in the pricing of natural gas which in turn affects the operational constraints of the facility. Due to the uncertainty, natural gas storage decisions can be viewed as stochastic in nature. Thus, the LP structure would not be equipped to 24

35 CHAPTER 2: Literature Review fully capture all the characteristics of the process and a valuation produced by such a model would be deficient. In order to truly portray the operational decisions of a natural gas storage system, a stochastic model would need to be incorporated, requiring stochastic optimization methods in order to solve for the value of the facility. Section 2.2 will discuss the use of Approximate Dynamic Programming (ADP) as a stochastic optimization method and evaluate its valuation capabilities in comparison to the method and results produce by linear optimization. 2.2 Approximate Dynamic Programming (ADP) Approach Due to the uncertainty and stochastic nature of natural gas storage decisions, scholars such as Lai et al. (2008), gravitated toward stochastic dynamic programming (SDP) as the natural approach to solving the storage valuation problem. Solving dynamic optimization problems is rooted in the application of Bellman s equation which solves simpler sub-problems, computing the value of a decision at a certain point in time in terms of the payoff of the current decision and the value of the remaining decisions within the problem that result from the prior decisions. Bellman s equation for stochastic problems can be described as V t s t, a t = max C x, a + γe V t+1 s t+1, a t+1 s t, a t t T, where C(x, a) is the contribution made at time t from taking action a while in state x, and γ discounts the expected value of the next state (Puterman 1994). Storage value is determined by the expected cash flow generated by the operational decisions. These decisions are dependent on the current operational characteristics of the storage facility in addition to the market conditions at the 25

36 CHAPTER 2: Literature Review time. Given the decision to inject or withdraw x and the operational and market characteristics that make up the state at time t, the value of storage can be expressed by the following Bellman s equation V t X t, F t = arg max CF t x, S t + e rdt E V t+1 X t+1, F t+1 F t t T, subject to W t x I t, X t+1 = x + X t, 0 x X max, CF t f, S t = f S t, where X t = the inventory level at time t with X max being total capacity x = the amount of flow in any given period t W t = the maximum withdrawal capacity at time t I t = the maximum injection capacity at time t s t = the gas price at time t which is determined by a specific stochastic process F t = the forward curve at time t CF t = the cash flow at time t 26

37 CHAPTER 2: Literature Review The solution approach for solving a discrete SDP is defined by backward induction, which can be implemented by first considering the last time a decision is made and choosing what to do in any situation at that time. Using this information, one can then determine what to do at the second-to-last time of decision. This process continues backwards until one has determined the best action for every possible situation. Though it appears an easily obtained solution can be reached, the presence of the forward curve vector, F t, within system state that complicates the ability to reach a solution. F t is a vector of forward prices at time t for all the futures contracts that mature at various times in the future. Over 72 contracts are traded in the futures market at one time, yet utilizing even a fraction of them within the state variable subjects the problem to the curse of dimensionality. This means that the high dimensionality of the state space and the manipulation of exponentially large volumes of information render the solution unfeasible by backwards induction. To combat this problem Lai et al. (2008) develops a technique based on ADP methods to value the storage of natural gas and renders the high-dimensional model of the forward curve more pliable. This approach transforms the intractable stochastic dynamic program model of the storage problem into a manageable lower dimensional Markov decision process. The approach to reducing the computationally intractable SDP model is to develop an approximate model using information reduction. Lai et al. (2008), removes price related state variables from the state definition of the exact model. 27

38 CHAPTER 2: Literature Review Though the amount of variables removed may fluctuate based on the solver s discretion, it is important to note that the more price-related variables are removed, the easier it will be to solve efficiently. The contraction within the ADP technique occurs by computing the optimal value function by conditioning on the possible values of the next month s futures price and the next months futures price at t = 0, instead of conditioning on the whole forward curve. So now the vector F t, is composed of two scalar variables, F t,t+1 and F 0,t+1. The state variable now includes the current inventory, spot price, and only two variables which are associated with the forward curve within each period t. Thus, the ADP model of the problem is vastly reduced from the previous model and is given by V t ADP X t, F t = E v t ADP X t, F t,t+1 F 0,t+1 t T, where, ADP v t X t, F t = arg max CF t x, S t + e rdt ADP E V t+1 X t+1, S t+1 F t,t+1. The conditioning on the next month s futures price at time 0, F 0,t+1, can be seen in the first equation and yields the expected value of the value function displayed by the second equation, which is conditioned on the next month s futures price at time t, F t,t+1. It is reported that the implementation of this ADP model can generate values that on average are 97% of optimal storage value, while the LP model of Section 2.1 reports a larger underestimate of the value of storage with an average of 75% (Lai, Margot and Secomandi 2008). It appears that ADP dominates the LP model in terms 28

39 CHAPTER 2: Literature Review of the actual valuation of the storage facility. However, looking at cpu run time as an alternative indicator, ADP can be considered as a suboptimal policy with extensively higher cpu requirements, averaging 250 cpu seconds compared to the.0038 cpu seconds required by the LP model. It is evident that despite the improvements in the actual valuation of natural gas storage, greater computational burdens are undertaken to produce such results. The fact that ADP solution methods are considered slower and less efficient, little impression is made on industry practitioners. Thus, it is important to find a method that will best balance valuation quality and computational efficiency. Chapter 3 will discuss the stochastic optimization method known as optimal learning, focusing on the stochastic search method known as the Knowledge Gradient Policy. We will utilize this process in order to find a valuation method for natural gas storage that deploys stochastic optimization methods lacking in the LP model, yet will yield less computational burdens found in the ADP model. 29

40 CHAPTER 3 Knowledge Gradient Methods Within the natural gas industry, decisions for the operation of a storage facility are based on policies which may be simple or complex in nature but all of which are based on heuristics. Letting the problem of valuing a natural gas storage facility be defined as computing the expectation of these operational decisions based on such a policy, we can define the problem within the framework of the stochastic optimization method known as optimal learning. The idea encompasses a stochastic search over policies which dictate the operational decisions of the storage facility. We assume the policies are composed of a set of tunable parameters where a range of suitable alternatives for the parameters will be established. The method is defined by the sequential measurements of the alternatives that estimate each policy s performance. Typically, a budget of measurements is spread over a set of alternatives. The goal is to choose the alternatives, and subsequently the policy, which will produce the best performance. Within the scope of our problem, the

41 CHAPTER 3: Knowledge Gradient Methods performance we wish to optimize is profit produced by the utilization of the storage facility and subsequently the value of the storage facility. The problem described above is considered a ranking and selection problem. A policy which has had much success within the problem class is known as the Knowledge Gradient policy (KG). First developed by Gupta & Miescke (1996) and later analyzed by Frazier et al. (2008), the knowledge gradient policy is a myopic policy that strives to solve ranking and selection problems by choosing alternatives that maximize the amount of knowledge obtained by a single measurement in an effort to obtain an improved solution with the new knowledge. This is done by looking one time step into the future when making a measurement decision. This chapter will not only discuss the Knowledge Gradient policy but also extensions made to the concept. This includes the correlation of measurements which applies to our problem of optimizing the natural gas policies where the measurement of one policy provides information regarding another. A new method within the realm of correlated measurement problems is also discussed within the chapter and will be the means of obtaining the valuation solution to the natural gas storage problem. 3.1 Knowledge Gradient with Independent Measurements To visualize the idea of the knowledge gradient policy we first consider problems with independent measurements. This means when measuring alternative x, no information is gained for the value of x x. In this setting, the true values of the alternatives are distributed by 31

42 CHAPTER 3: Knowledge Gradient Methods Y ~ Ν(μ x, σ x 2 ), where the parameters are unknown (Powell and Frazier 2008). However, assuming that at iteration n our state of knowledge is given by S n = (μ n, β n ), where β n is the precision of alternative x defined by β n = 1 ς 2,n, a prior predictive distribution of Y for each alternative x can be obtained which is n normally distributed with mean μ x and variance ς 2,n x (Frazier and Powell 2008). If measurements were to cease after n iterations, the best option would yield a solution represented by V n S n = max x χ μ n x. If measurements continued, the next state of knowledge would be S n+1 (x), where x has been chosen to be measured, producing y x n+1, the observation of the n+1 measurement. Defining ε as the noise of a measurement, a generalized description for an observation can be written as y x = Y x + ε. With a new observation, a posterior predictive distribution for Y can be determined with mean μ x n+1 and variance 1 β x n +1 (Frazier and Powell 2008). Using Bayes rule, the updated values for μ x n+1 and β x n+1 are given by 32

43 CHAPTER 3: Knowledge Gradient Methods μ x n+1 = β x n μ x n + β ε μ x n+1, μ x n, if x n = x oterwise, β x n+1 = β x n + β ε, β x n, if x n = x oterwise. This allows for the computation of the best solution after n+1 measurements represented as V n+1 S n+1 n+1 (x) = max x χ μ x. n+1 Going back to the perspective at iteration n, it is important to note that μ x n is a random variable, while μ x is not. The objective is to choose an alternative x that maximizes the incremental increase between the best value at n and what is believed to be the best value at n+1. This is given by v KG,n = max x χ E Vn+1 S n+1 x V n S n S n, (3.1) which is known as the value of the knowledge gradient (Frazier, Powell and Dayanik 2008). Equation 3.1 can be interpreted mathematically as a gradient of V n S n with respect to the measurement chosen which is maximized over the knowledge gained from that measurement. The knowledge gradient policy can then be expressed by X KG,n = argmax x χ E[Vn+1 S n+1 (x) V n S n S n ]. A simple example of the knowledge gradient policy can be seen in Figure 13. Here, five choices are considered, where the estimate mean of choice 4 appears to be the best solution. Looking at the distribution of alternative 5, there is a distinct 33

44 CHAPTER 3: Knowledge Gradient Methods probability that choosing alternative 5 will produce a value greater than the current, best witnessed in part of the distribution curve that exceeds choice 4. If this occurs, V n+1 will increase; or be 0 if it does not. The expected increase of V n+1 is the knowledge gradient v KG. Figure 13 - Illustration of Knowledge Gradient (CASTLE Laboratory ) 3.2 Knowledge Gradient for Correlated Beliefs Many problems are characterized as having one measurement providing information about what might be observed from other measurements. This is known as correlated beliefs, and the knowledge gradient concept has been extended by Fraizer (2009) for greater accessibility to this problem class, naming the policy the Knowledge Gradient for Correlated Beliefs (KGCB). The move from independent to correlated measurements involves a transition to working with vectors of means and covariance matrices. In this setting, the values of the alternatives are distributed by 34

45 CHAPTER 3: Knowledge Gradient Methods Y ~ Ν(μ, Σ), where μ is a vector of means with element μ x, and the means of the alternatives are represented by x. Let Σ be the covariance matrix with element Cov(x, x ) (Frazier, Powell and Dayanik 2009). Assuming that at iteration n our state of knowledge is S n = (μ n, Σ n ), the knowledge gradient with correlated beliefs will act similarly to its independent counterpart by selecting an alternative that is expected to increase the current best estimate given by v KG,n = argmaxe[max x μ n+1 x max x μ n x S n, x n = x]. x χ After taking a measurement of alternative x, y x n+1 is observed as the n+1st measurement, where the observation is normally distributed with mean Y x and variance λ x which is defined as λ x = 1 β ε. Transitioning to the new state S n+1 = (μ n+1, Σ n+1 ) obtains a posterior distribution for our beliefs with parameters μ n+1 and Σ n+1 which are calculated from the Bayesian updating formulas given by μ n+1 = μ n + y n +1 μ n x n Σ n e x, λ x +Σ x,x Σ n+1 = Σ n + Σn e x e x T Σ n λ x +Σn, x,x 35

46 CHAPTER 3: Knowledge Gradient Methods where e x is a column vector of zeros except for a single 1 where x is indexed (Frazier, Powell and Dayanik 2009). For simplicity, μ n+1 can be written as μ n+1 = μ n + ς(σ n, x n )Z, where Z is a standard normal random variable and ς Σ n, x n = Σn e x λ x +Σn x,x. Integrating this into the v KG formula for the KGCB, it can be restated as v KGCB,n = maxe max[(μ n x χ x x + ς x Σ n, x n )Z S n, x n = x max μ n x x χ where the policy will choose the alternative which maximizes this value (Frazier, Powell and Dayanik 2008). In Figure 14, the implementation of the KGCB policy can be seen in a series of graphs that juxtapose a graph of the measurements and the value of the knowledge gradient, v KGCB,n, at three distinct points in time. The first shows the first measurement and its effect on the value of the knowledge gradient. Whenever a point is measured, knowledge is gained not only about the measurement but also about surrounding points which are seen by the drop in the knowledge gradient at the point of measurement and its surrounding area. This is due to the correlation of the measurements. The policy will continue to select areas of high uncertainty to measure. 36

47 CHAPTER 3: Knowledge Gradient Methods Figure 14 Knowledge Gradient with Correlated Beliefs at Measurements 1 (CASTLE Laboratory ) Figure 15 portrays the fifth measurement where the policy has chosen to measure a point within the middle region of the surface due to the uncertainty of the region. Knowledge is once again gained not only about the specific point we measured, but also the region surrounding the point. Figure 15- Knowledge Gradient with Correlated Beliefs at Measurement 5 (CASTLE Laboratory ) The graphs in Figure 16 displays the final estimate of the surface based on all the measurement decisions of the policy. 37

48 CHAPTER 3: Knowledge Gradient Methods Figure 16 - Knowledge Gradient with Correlated Beliefs Estimated Surface (CASTLE Laboratory ) 3.3 Knowledge Gradient with Non-Parametric Estimation The KGCB policy assumes that we have prior knowledge regarding the covariance structure. For problems outside of this class, Barut (2010) introduces the Knowledge Gradient with Non-Parametric Estimation (KGNP), a sequential learning method that aggregates a set of estimators derived from multiple kernel regression with different bandwidths. Unlike KGCB, this policy does not assume prior knowledge of the covariance structure Kernel Estimation Kernel estimation methods are at the core of KGNP, and thus this section strives to provide a brief overview in order to illuminate the KGNP procedure discussed in Section Consider an unknown function where the parametric form is unclear. We have observed the following points related by the function, x 1, y 1 (x n, y n ) and define the conditional expectation of Y given X as 38

49 CHAPTER 3: Knowledge Gradient Methods m x = E Y X = x]. This allows us to use the common procedure of estimating the value of the function by weighing nearby points, m x = n i=1 w i y i n i=1 w i. m x can be estimated as a locally weighted average, using a kernel as a weighting function (Nadaraya 1964). This can be seen in the Nadaraya-Watson estimator m x = n i=1 K (x x i )y i n i=1 K (x x i ), where h is a positive number known as the bandwidth, which is dictated at the discretion of the modeler, and K is the kernel function where K x x i = 1 K(x x i ). Kernel functions are always positive and chosen based on the bounded region they support. A compilation of kernel functions frequently applied as estimators are exhibited in Table 4 and are graphically displayed in Figure

50 CHAPTER 3: Knowledge Gradient Methods Table 4- Commonly Used Kernel Functions Kernel Function K(u) Uniform K u = { u 1 Triangular K u = 1 u 1 { u 1 Epanechnikov K u = u2 1 { u 1 Quartic(biweight) K u = u2 2 1 { u 1 Triweight(tricube) K u = u2 3 1 { u 1 Guassian K u = 1 1 2π e 2 u2 Cosine K u = π 4 cos π 2 u 1 { u 1 Figure 17 - Plot of Kernel Functions K(u) 40

51 CHAPTER 3: Knowledge Gradient Methods Picking a suitable kernel function for estimation is simplified in Hardle et al. (2004) who shows that different kernel functions share similar properties in estimation rendering the choice of function negligible. The challenge now reverts to the selection of the bandwidth size. The bandwidth can be thought of as the width of a window centered at the data point and giving weight to any points located in the window. In KGNP, explained in Section 3.3.2, the bandwidth selection will be established by fixing a set of bandwidths and reweighing the estimates proposed by each according to the accuracy of their measurements (Barut and Powell 2010). x to be KGNP Procedure and Derivation We begin by establishing the value for the knowledge gradient for alternative v KG,n = max E[max x x χ μ n +1 x max x μ n x S n, x n = x]. (3.2) In the KGNP approach, the estimate of μ n+1 x is formed by the weighting of estimators, μ k,n+1 x, where k denotes a different kernel estimation method and/or bandwidth within a set of kernel estimation methods denoted by K. The estimator, μ n+1,k x, can be written as μ x n+1,k = β x Κ.k x,x x 0,n + Κ.k x,x n (β n 0,n x χ x n x n +(βx ε n y n x n +1 ) n, x χ β x +1 Κ.k (x,x ) where h represents the bandwidth used within kernel regression method k. The function K(.,.) is the kernel weight used to obtain the local estimate which will be different for every bandwidth (Barut and Powell 2010). 41

52 CHAPTER 3: Knowledge Gradient Methods The total precision obtained by making measurement x is given as k A n+1 x, x n = β n+1 x Κ.k (x, x ) + β ε xn Κ.k x, x n. x χ Thus, μ x n+1,k can be expressed as μ n+1,k x = μ n,k x + β x ε n Κ.k x, x n k A n+1 x, x n μ xn μ x n,k + ς x, x n, k Z, where Z is a standard normal and n ς x, x n, k = ( ς 2 xn + ς 2 xn ) β x ε n Κ.k x, x n k x, x. n A n+1 Now μ n+1 x, given x n at time n, becomes μ n+1 x = k w x n+1,k 1 β x ε n Κ.k x, x n k A n+1 x, x n μ x n,k + μ xn k w x n+1,k β ε xn Κ.k x, x n k A n+1 x, x n + Z w x n+1,k ς x, x n, k k, where the estimates produced by k and h are weighed according to w (Barut and Powell 2010). However, because future weights w n+1 are not known, predictive weights are used given by w n,k n,k x x = ( ς 2 x + δ k x ) 1 kεk 1, where 42

53 CHAPTER 3: Knowledge Gradient Methods ς x n,k 2 = Var μ x n+1,k = (β n x χ +1 x Κ.k x,x ) 2 Var (x 0,n ) n = ( x χ β x +1 Κ.k (x,x )) 2 β n x χ +1 x Κ.k x,x 2 n ( x χ β x +1 Κ.k (x,x )). 2 The variable δ x k is the estimated bias of the kth kernel of alternative x. Kernels with higher bias get less weight among the kernels exhibiting high variance (Barut and Powell 2010). Integrating our result for μ n+1 x with Equation 3.2, we are able to determine the knowledge gradient for KGNP to be v KG,n = E max x χ a n x (x) + b n x (x)z S n max μ x n, x χ where a x n x n = k w x n+1,k 1 β x ε n Κ.k x, x n k A n+1 x, x n μ x n,k + μ xn k w x n+1,k β ε xn Κ.k x, x n k x, x, n A n+1 and b x n x n = w x n+1,k ς x, x n, k. k This method will be utilized in determining optimal parameters within the policies developed in Chapter 5 for the optimal utilization of a storage facility. Results concerning the policies performance will be presented in Chapter 6 43

54 CHAPTER 4 Pricing Models Decisions to inject and withdraw from a storage facility rely heavily on the spot and forward prices of natural gas. It is important to accurately simulate these pricing processes in order to accurately induce appropriate operational responses of the salt cavern storage facility. The modeling process consists of considering the market s qualitative properties, selecting a model that matches the properties and estimating the parameters based on historical data. From these an accurate model of the market can be produced. This chapter will demonstrate the modeling processes used to simulate the spot and futures pricing process for the natural gas market, utilizing historical prices reported by NYMEX at Henry Hub in Louisiana. 4.1 Geometric Brownian Motion An industry standard for modeling price evolution is the use of geometric Brownian motion (GBM). GBM has been extensively studied, and its properties are well known, including its ability to generate only positive random numbers which is

55 CHAPTER 4: Pricing Models important for financial applications when the numbers represent prices (Eydeland and Wolyniec 2003). The GBM process in its standard form is expressed as ds t S t = μ dt + ςdw t. Here, ds t signifies a random movement of the spot price S t over a small interval of time dt. The drift or rate of return of an asset as well as the volatility of the asset are constants represented respectively by μ and σ. Lastly, dw t denotes the increments of standard Brownian motion over the same period of time. Increments of dw t are normally distributed with mean zero and standard deviation dt. W t ~ Ν 0, dt 4.2 Spot Price Process Though excellent for a variety of financial applications, geometric Brownian motion in its standard form is ill suited to describe the evolution of energy prices. Particularly, the natural gas market requires a model which describes the presence of mean reversion within the spot prices. This attribute is present due to the pressures of supply and demand on the price caused by the seasonal use of natural gas. High demand during the winter months will put upward pressure on the price of natural gas, and similarly the limited demand and thus increased supply of natural gas during the spring will place downward pressure on the price. Amongst these high and low swings, the price is centered around an average that the market wants to regress to as it moves further away. The mean reversion within natural gas prices can be captured in the stochastic process known as the Ornstein-Uhlunbeck 45

56 CHAPTER 4: Pricing Models process. Using variables related to the topic of natural gas, the process can be described by the following stochastic differential equation ds t S t = α(μ t log S t ) dt + ς t dw t. (4.1) Here, μ(t) is the mean reverting level or price, and α represents the strength of the mean reversion or the rate at which the price will try to revert back to the mean reverting price. The volatility, ς t, is now a function of time due to its dependence on the changing mean reverting level. The process of obtaining each parameter will be discussed in later sections of this chapter. The other variables retain their identity from the standard GBM process (Eydeland and Wolyniec 2003). From Equation 4.1, a closed form solution for the distributions of the logarithms of prices can be obtained. Utilizing a simple change of variables from which we obtain S t = e X t (4.2) X t = log S t. Applying Itô s Lemma, the process can be written as dx t = α μ t X t dt + ς t dw t, where dx t is the change in the log price of natural gas. Introducing a new variable Y t = e αt X t (4.3) and again utilizing Itô s Lemma, the process of dy t can be expressed as dy t = αμ t e αt dt + ς t e αt dw t. 46

57 CHAPTER 4: Pricing Models Since dy t is a normal random variable for every t, the summation of these increments from the current time to the future time will produce the random variable Y t+δt ~ Ν Y t+δt + μ t e αδt, ς t e 2αΔ t 2α. Hence, by the definition of Y t given in Equation 4.3, X t+δt ~ Ν e αδt X t + μ t (1 e αδt ), ς t (1 e 2αΔt ) 2α (4.4) which is a closed form solution for the distribution for the logarithms of natural gas spot prices (Eydeland and Wolyniec 2003). However, before simulations can be implemented, parameters α, μ(t), and σ(t) must be determined. The parameters will be estimated using the historical data of Henry Hub Natural Gas spot prices from the years , which can be seen in Figure

58 Price ($/MMBtu) CHAPTER 4: Pricing Models Figure 18 - Henry Hub Historical Natural Gas Spot Prices (Enstor 2009) Spot Price /30/03 02/27/05 04/28/06 06/27/07 08/25/08 10/24/09 Date (mm/dd/yy) Strength of Mean Reversion (α) The parameter α is the strength or rate of mean reversion, which is the rate at which the price will move toward a certain level. To estimate α, a linear regression is implemented on the historical spot prices. Here, the log return of the spot prices, log ( S t+1 ), are regressed against the log of the ratio between the S t historical average, μ, of the data and the spot price, log ( μ S t ). The slope of this regression is the rate of mean reversion. For this data set, α was computed to be

59 CHAPTER 4: Pricing Models Exponential Smoothing The mean reverting level for this process, μ t, is time dependent due to the assumption that the long-term mean will alter over the course of many years. From Figure 18, the average price of natural gas in 2009 appears to be approximately $4.00 per MMBtu, while years prior show an average of $7.00 or higher. Thus, the technique of exponential smoothing is utilized to capture the movement of this longterm mean. Similar to a moving average, exponential smoothing assigns exponentially decreasing weights over time instead of assigning equal weights to past observations allowing current measurements to hold more influence. A simple expression for the exponential smoothing algorithm is μ t = βs t + (1 β)μ t 1 (4.5) where β is the smoothing factor between [0,1] (Brown and Meyer 1960). Thus, the smoothed statistic, in this case the mean reverting level, μ t, is a function of the weighted average of the new information receive, the spot price S t, and the previous smoothed statistic, μ t 1. Tuning β will change the smoothness of the observations produced form the process. The smaller the β the smoother the observations, and conversely the larger the β the more the observations will track the noise of the original data. For the Henry Hub Natural Gas Spot Prices a β of.002 was chosen in order to capture a gradual shift in the long-term of the spot price. The implementation of exponential smoothing can be seen in Figure19. 49

60 Price ($/MMBtu) CHAPTER 4: Pricing Models Figure 19 - Natural Gas Spot Prices with Exponential Smoothing, Beta = Spot Price Exponential Smoothing Price /30/03 02/27/05 04/28/06 06/27/07 08/25/08 10/24/09 Date (mm/dd/yy) Volatility Earlier it was shown that the distribution of X t, the logarithms of prices, had a standard deviation of ς t = ς t (1 e 2αΔt ) 2α (4.6) where ς t is the volatility seen in the pricing process described by Equation 4.1. Rearranging Equation 4.6, the volatility can be written as a function of the standard deviation of the logarithm of S t. 50

61 CHAPTER 4: Pricing Models ς t = ς t 2α (1 e 2αΔt ) (4.7) The value of ς t can be estimated using either exponential smoothing similar to the form in Equation 4.5 or from a rolling window of historical spot price data by applying the formula ς t = 1 m 1 m i=1 log S i log S i 1 Δt 1 m m i=1 log S i log S i 1 Δt 2 where S i represents the price of natural gas at the closing of each day, which is measured from historical data during a window of m time periods prior to the current time period (Eydeland and Wolyniec 2003). Using this estimate in Equation 4.7, a time dependent volatility for the pricing process can be determined. Simulation Results Having selected a mean reversion model and estimated the appropriate parameters based on the historical data, the model expressed in Equation 4.1 can be used to simulate the evolution of the spot prices for natural gas. Figure 20 illustrates several sample price paths constructed from the simulations of a year of natural gas spot prices. 51

62 Price ($/MMBtu) CHAPTER 4: Pricing Models Figure 20 - Sample Paths for 1-Year of Simulated Natural Gas Spot Prices /01/10 02/20/10 04/11/10 05/31/10 07/20/10 09/08/10 10/28/10 12/17/10 Date (dd/mm/yy) Path 1 Path 2 Path 3 Path 4 Path Forward Price Process In order to implement a forward pricing process, it is important to utilize the simple relationship between the spot price, denoted by S t, and the price of the forward contract maturing at T, denoted by F t,t. F t,t = E S T S t This equality states that the forward price is the expectation of the spot price at time T given the information available about the spot price at time t. The expectation is taken over all the paths of the mean reverting process defined in Equation 4.1. Computing the forward price depends heavily on the spot price process established 52

63 CHAPTER 4: Pricing Models in Section 4.2. To begin, the change of variables established in Equation 4.2 is utilized and allows the forward price to be expressed as F t,t = E e X T e X t. From Equation 4.4, it is known that X T is normally distributed with mean μ = X t e α(t t) + μ T 1 e α T t (4.8) and variance ς 2 = ς t 2 (1 2α e 2α T t ). (4.9) The parameters include μ T which is defined as the mean reverting level of the spot price at time T. This will be approximated by μ t due to the assumption that the mean reverting level at time T will on average be the same as at time t. The computation for μ t, as well as for the mean reversion rate α and the volatility ς t will have been conducted during the spot pricing process by the techniques outlined in Section 4.2 (Eydeland and Wolyniec 2003). Having a closed form solution for the distribution of X T and the estimates for the parameters, the expectation can be solved using the moment generating function of a normal random variable yielding the price of a forward contract to be F t,t = E e X T e X t = e μ +1 2 ς 2. Thus, the calculation of the forward price at time t will immediately follow the result of the computation of the spot price by Equation 4.1 for the same period (Eydeland and Wolyniec 2003). 53

64 Price ($/MMBtu) CHAPTER 4: Pricing Models In Figure 21, a single sample path of the simulation that utilizes the forward pricing process for a year s worth of month-ahead contracts is seen with the respective spot price simulation. Figure 21 - Simulated Month Ahead Forward Prices for One Year Month-Ahead Price Spot Price /01/10 02/20/10 04/11/10 05/31/10 07/20/10 09/08/10 10/28/10 12/17/10 Date (mm/dd/yy) From Equation 4.9 it can be seen that the volatility from the forward price simulation will converge to zero as time approaches T. This allows for the convergence of the forward price to the spot price at time T which is characteristic behavior for the forward price. This relationship is given in Equation 4.10 and can be seen clearly in Figure 22 which displays simulated forward and spot prices for one month. lim t T F t,t = S T. (4.10) 54

65 Price ($/MMBtu) CHAPTER 4: Pricing Models Figure 22 - Month Ahead Forward Prices with Spot Prices for One Month One Month Forward Simulation Month Ahead Price Spot Price /01/10 02/06/10 02/11/10 02/16/10 02/21/10 02/26/10 Date (mm/dd/yy) 55

66 CHAPTER 5 Model and Policies Consider a natural gas commodities trader holding a one year salt cavern storage contract. The contract details are presented in Table 5. The trader is participating in the physical trading of natural gas and will use the contracted storage facility to hedge his activity within the natural gas market. This allows him to make the decision of buying and injecting gas into the facility, withdrawing and selling the gas, or storing the gas for later use. The trader wishes to maximize the terminal profit from trading natural gas at the end of his storage contract. This requires the trader to make his decisions to inject, withdraw, or store based on not only the price of natural gas but also the operational capabilities of the storage facility. The decisions of the trader will generate a series of cash flows that when discounted will ascertain the terminal profit as well as the value of the storage facility at a given point in time. This chapter will model these decisions and operations of the storage facility as well as describe the policies used to direct the decisions.

67 CHAPTER 5: Model and Policies Table 5 - Salt Cavern Storage Contract (Enstor 2009) Term 01/01/ /31/2010 Type Depth Maximum Working Gas Capacity Initial Working Gas Capacity Maximum Injection Rate Maximum Withdrawal Rate Salt Cavern 3500 ft 7.5 Bcf 0 Bcf.3 Bcf/day.5 Bcf/day Fuel Injection Loss 1.5% Maximum/Minimum Facility Pressure Compressor.85 PSI/foot.2 PSI/foot 25,000 horse power 5.1 Modeling Assumptions A series of assumptions were made in creating this model in order to achieve simplicity and ease in its comprehension. They are as follows: Assumption 1: We consider the scope of the trading activity negligible compared to overall market activity, meaning a decision to inject or withdraw natural gas in order to participate in market transactions will hold no influence over its price on the spot and futures market. For example, on April 2, 2010 over 147,000 forward contracts were traded equating to 1,470,000,000 MMBtu of natural gas (CMEGroup 2010). Considering that the maximum that could be injected or withdrawn in one day by the storage facility in Table 5 is 300,000 and 500,000 57

68 CHAPTER 5: Model and Policies MMBtu s, only.021% and.034% of the traded volume, it can be established that a single gas storage facility is a small player within the market. Assumption 2: Decisions will be restricted to a maximum of once a day. This is due to the fact that simulations of the spot and forward price of natural gas described in Chapter 4 are limited to producing daily evolutions in price. This cuts the complexity of producing finer discretizations of the change in price. This restriction in the frequency of gas prices results in the ability to only make decisions as the price changes, that being once a day. It would be illogical to make multiple decisions on stagnant information. Assumption 3: The physical delivery of natural gas purchased or sold on the spot market is subject to the rules and regulations of NYMEX and must take place the day after the market transaction occurred (CMEGroup 2010). A spot purchase of 10,000 MMBtu purchased on Monday, April 12, 2010 must be physically delivered to the buyer on Tuesday, April 13, Assumption 4: The physical delivery of natural gas purchased or sold on the futures market is subject to the rules and regulations of NYMEX and takes place on the month the contract matures and is delivered evenly throughout each day of the month (CMEGroup 2010). Therefore, it is assumed that 1/30 of the contract is delivered each day of the maturity month, given that a month contains 30 days. 5.2 Problem Set-Up and Terminology Due to the availability of daily pricing data for natural gas, the problem will be modeled in terms of days and months. If D is the set of days and M is the set of 58

69 CHAPTER 5: Model and Policies months, then D = {1,2,3,..,360 and M = {1,2,3,,12. The model is constructed according to a six element structure that highlights the decision variables, exogenous information, state variables, transition functions, contribution function, and objective function of the problem. This section outlines this framework detailing each of the elements in accordance with the problem at hand Parameters X max - upper volume limit of working gas in the storage facility (Bcf) d- depth of cavern in feet I r max - the maximum injection rate at the storage facility (Bcf/day) r W max - the maximum withdrawal rate or deliverability at the storage facility (Bcf/day) I r min - the minimum injection rate at the storage facility (Bcf/day) r W min - the minimum withdrawal rate or deliverability at the storage facility (Bcf/day) p max - the maximum pressure within the storage facility (PSI/foot) p min - the minimum pressure within the storage facility (PSI/foot) C max - the maximum horsepower of wellhead compressor α- the percent fuel rate loss for injection at a storage facility γ- a risk-free discount factor 59

70 CHAPTER 5: Model and Policies Decision Variables and Constraints The decision whether to inject or withdraw is considered with respect to the prices of both the spot market and the forward market. Thus, this problem is composed of two decision variables x and y, which represents the volume of natural gas injected or withdrawn based on a transaction in the spot or forward market respectively. x d,d+1,m - the volume of natural gas injected or withdrawn on day d+1 from facility based on the transaction on spot market on day d and month m, (Bcf) x d,d+1,m > 0 if injected x d,d+1,m = 0 if no injection or witdrawal occurs x d,d+1,m < 0 if witdrawn y m,m - the volume of natural gas to be injected or withdrawn from facility in month M based on the transaction in futures market that took place on day d during month m, (Bcf) y d,m,m > 0 if injected y d,m,m = 0 if no injection or witdrawal occurs y d,m,m < 0 if witdrawn The sign of a decision corresponds to the increase or decrease in capacity. The positive decision variables, x d,d+1,m, y m,m, will represent injections of natural 60

71 CHAPTER 5: Model and Policies gas while negative decision variables will reflect a withdrawal of natural gas from the facility. The volume of gas injected or withdrawn per day is constrained by the operational capabilities of the storage facility. Thus, the decision of how much to inject and withdraw is bounded by the withdrawal capacity and injection capacity of the facility at time t, where I d+1 and W d+1 are considered as system-state variables and are defined in Section x d,d+1,m + y d,m 1,m 30 I d+1 x d,d+1,m + y d,m 1,m 30 W d+1 It is also important to note the division of the forward decision variable to reflect the daily distribution of a forward contract throughout the month as stated in Assumption 4. An additional constraint arises due to Assumption 3 and 4 where all natural gas transactions must be settled by the delivery date. For the spot market this is the day following the transaction, the operation time of injection and withdrawal must take place within the 24 hour time block of the delivery date. Thus, the decision variables are bounded by the daily injection and withdrawal rate, r I and r W, for the specified day of delivery. 1 I x d,d+1,m + 1 I r d +1 r d+1 y d,m 1,m 30 1 wen x d,d+1,m, y d,m 1,m > 0 61

72 CHAPTER 5: Model and Policies 1 W x d,d+1,m + 1 I r d +1 r d+1 y d,m 1,m 30 1 wen x d,d+1,m < 0, y d,m 1,m > 0 1 I x d,d+1,m + 1 W r d +1 r d+1 y d,m 1,m 30 1 wen x d,d+1,m > 0, y d,m 1,m < 0 1 W x d,d+1,m + 1 W r d +1 r d+1 y d,m 1,m 30 1 wen x d,d+1,m < 0, y d,m 1,m < 0 The rates within these constraints are considered System State variables and will be discussed in Section Exogenous Variables s d+1,m - the change in the natural gas spot price at day d, month m ($) F d+1,m,m+1 - the change in the natural gas futures price at day d and month m for physical delivery during month m+1 ($) System State X d - the volume of working gas at the storage facility at day d. 0 X d X max, (Bcf) I d - Injection Capacity at day d, (Bcf) W d - Withdrawal Capacity at day d, (Bcf) r d I - the injection rate at the storage facility at day d, (Bcf/day) r d W - the withdrawal rate or deliverability at the storage facility at day d, (Bcf/day) p d - the pressure within the storage facility at day d, (PSI) 62

73 CHAPTER 5: Model and Policies Together, these variables compose the state of the system which provides the information needed to make decisions whether to inject or withdraw natural gas. S d X d, I d, W d, S d, F d,m,m+1, r d I, r d W, p d - the state of the system/storage at day d, Transition Functions Transition of Market Prices: s d+1 = s d + s d+1 F d+1,m,m+1 = F d,m,m+1 + F d+1,m,m+1 Transition of Storage Capacity and Operational Elements: W d+1 = X d+1 + x d,m + y d,m 1,m 30 I d+1 = X max W d+1 X d+1 = X d + α I x d,m + α I y d,m 1,m 30 X d + α I x d,m + y d,m 1,m, 30 X d,, if x d,m, y d,m 1,m > 0 if x d,m > 0, y d,m 1,m < 0 if x d,m = 0, y d,m 1,m = 0 X d + x d,m + α I y d,m 1,m, if x 30 d,m < 0, y d,m 1,m > 0 X d + x d,m + y d,m 1,m, if x 30 d,m, y d,m 1,m < 0 I r d+1 = r I max I r min X max I X d+1 + r max r W d+1 = r max W W r min X max W X d+1 + r min 63

74 CHAPTER 5: Model and Policies p d+1 = p max p min X max X d+1 + p min C d+1 = C max p max p d+1 Transition of Operational Costs: spot,i = C d ( 1 I 24 x d ) s d / c d+1 r d+1 spot c,w d+1 =.01 x d forward,i = C d ( 1 c d+1 I r d+1 24 y d,m 1,m 30 ) F d,m 1,m / forward c,w d+1 =.01 y d,m 1,m Contribution Function The reward for an action is denoted by r x d,m, y d,m 1,m, s d where x d,m and y d,m 1,m are associated with the operational decision to injection, withdraw or do nothing during day d. Rewards for injections will be negative quantities considering injections are equated with commodity purchases and cash outflows. Withdrawals will have positive rewards corresponding with commodity sales and cash inflows. 64

75 CHAPTER 5: Model and Policies r x d,m, y d,m 1,m, s d α I s t x d,m c t spot,i + α I F t,t y d,m 1,m 30 α I s t x d,m c t spot,i if x d,m, y d,m 1,m > 0 + F t,t y d,m 1,m 30 ifx d,m > 0, y d,m 1,m < c t forward,i 10 6 c t forward,w = 0 if x d,m, y d,m 1,m = 0 s t x d,m 10 6 spot c,w t + α I F y d,m 1,m t,t 30 if x d,m < 0, y m 1,m > 0 y d,m 1,m s t x d,m 10 6 spo c t,w t + F t,t 30 if x d,m, y d,m 1,m < c t forward,i 10 6 c t forward,w Objective Function The storage value can be computed by solving the following optimization function 360 V X i, Y i = max πεπ E d=0 γ r(s d, X π d ( S d ), Y π d (F d,m,m+1 )), where γ is a discount factor associated with the specific time. The objective function attempts to maximize the expectation of the sum of the discounted rewards over the year according to a policy π, which is determined from a set of possible values П. 5.3 Policies Speculators in the natural gas market aim to earn the maximum profit from the purchase and sale of natural gas. Thus, policies are derived from trading algorithms which aim at discerning a proper time and hopefully low price to buy and conversely a high price for which to sell. A common trading algorithm is swing 65

76 CHAPTER 5: Model and Policies trading where the asset is bought or sold at or near the end of an up or down price swing, or price trend. The determination of when these swings occur varies and the policies will explore a few of the swing trading techniques in order to best try and capture the trends of the natural gas market. Policies will also be tested which make use of the seasonality of prices and make decisions based at what point in the year is the decision being made. This section will describe all the policies tested in the model Swing Trading As stated earlier, swing trading endeavors to buy and sell an asset when the price approaches the end of a trend. For example, during a downward trend in price, a trader will aim to buy the asset at the bottom of the trend, riding the swing upwards and anticipating the sell of the asset at the top of price oscillations. Various swing trading strategies have been developed in order to anticipate when these swings occur. One simple system popular in the 1970s was Donchian s Four-Week Rule, which instructed the trader to buy when the price of an asset rose above the high of prior four weeks and sell when the price fell below the low of the same four weeks. Another technique stipulates a purchase of the asset after three consecutive highs and a sale following three consecutive lows. In order to anticipate the price oscillations of the natural gas market, a technique based on the percentage of price movement has been chosen. This technique removes actual price from the decision, moving away from strategies which base decisions on tangible price levels. A good image for this technique is a 66

77 CHAPTER 5: Model and Policies filter, where small price variations are removed from the decision process, leaving the algorithm less susceptible to false trends (Merrill 1977). A simple example of the strategy would be to base buy and sell decisions on price movements of 5% or better over a certain period of time. Figures 23 and 24 display the strategies which only identifies price changes of greater than 5% and 10% respectively. The red line displays the trend, and the max and min of these lines represent a sell and buy decision. The policies utilized in the model in Section 5.2, will incorporate both long term and short term windows of time independently for evaluating the price change as well as combine the two perspectives in order to gain greater accuracy in predicting market trends. The policies will also be tuned in order to discern the optimal percentage of price movement, as well as other parameters for which to base the buy and sell decisions on. This will be achieved through the various Knowledge Gradient methods described in Chapter 3. 67

78 Price ($/MMBtu) Price ($/MMBtu) CHAPTER 5: Model and Policies Figure 23 Swing Trend with 10% Filtering Day Figure 24 - Swing Trend with 5% Filtering Day 68

79 CHAPTER 5: Model and Policies Policy 0 Short-Term Speculation This policy concerns itself with earning the maximum profit by capitalizing on short-term price movements of natural gas. The policy will be utilizing swing trade methods where the change in price will depend on the pricing data from the prior three days. Thus, a decision will be made if the percent of change in price after three days breaches a particular level. The decision level will also be a percent, and may be set differently for the percent gain which stipulates a sell decision and the percent loss which indicates a sell decision. The variables of the policy can be explicitly expressed as ρ = the percentage level of price increase needed to instigate withdrawal φ = the percentage level of price decrease needed to instigate injection θ d = the percentage level of price increase or decrease at time d based on pricing data of last 3 days The last question remaining before being able to express the policy is how much natural gas should be injected or withdrawn when the criteria for the percentage change is reached. An assumption is made that the maximum amount of natural gas would be bought or sold in order to capitalize on the price which has matched the policy s criteria. The amount that can be purchased or sold is bounded respectively by the maximum capacity the facility could inject or withdraw on the I day of delivery d+1. It is expressed by the injection and withdrawal rates r d+1 and W r d+1 defined in Section Thus Policy 0 can be expressed in mathematical notation by 69

80 CHAPTER 5: Model and Policies I x d,d+1 = r d+1 1 { θd <φ r W d+1 1 {θd >ρ where x d,d+1 is the decision made on day d for the amount of natural gas to be injected or withdrawn at time d+1. Variables φ and ρ are tunable and obtaining their optimal values will be achieved through Knowledge Gradient Algorithms discussed in Section Policy 1 Long Trend Speculation This policy is identical to the policy described in Section 5.3.2, except the percentage change in price will be determined by taking into account the price four weeks prior to the current day instead of three days prior. The variables are the same and similarly φ and ρ will be tuned to represent the optimal operation of the policy Policy 2 Long and Short Trend Speculation The policy described here extends the policies represented in Sections and by encompassing a look at both long and short term trends. This is expected to enable the algorithm to better identify the transition between trends up and down having information on both the long and short market activity. θ d sort = the percentage level of price increase or decrease at time d based on pricing data of last three days θ d long = the percentage level of price increase or decrease at time d based on pricing data of last four weeks 70

81 CHAPTER 5: Model and Policies For this policy there will be four tunable parameters expressing certain criteria from which a decision to inject, withdraw, or do nothing will be instated. ρ s = the percentage level of price increase for short term trend φ s = the percentage level of price decrease for short term trend ρ l = the percentage level of price increase for long term trend φ l = the percentage level of price decrease for long term trend Based on the price at day d and the percentage change from four weeks and three days, a decision will be made. For example, if a certain price has a four week percent gain in price that surpasses ρ l as well as a three day gain that surpasses ρ s, then the assumption can be made that the price of natural gas is in a steady upward trend. Thus, the decision will be to do nothing in order to follow the trend upwards and withdraw to sell at an even higher price. This could be indicated when the four week percentage gain in price does not exceed ρ l, and the three day percentage is a gain that does not exceed ρ s or is a loss. These scenarios suggest the beginning of a downward trend, and thus a peak in natural gas price. A decision should be made to withdraw and try to capture the price peak before the downward trend gets too far into its run. Table 3 displays all possible scenarios surrounding the criteria and the resulting decisions. Once again, a decision to withdraw or inject would call for the sale or purchase of the maximum amount the facility could withdraw or inject on the day of delivery, d+1. 71

82 CHAPTER 5: Model and Policies Table 6 - Mixed Trend Decisions # 4 Week Trend θ d long >ρ l θ d long <φ l 3 Day Trend θ d sort >ρ s θ d sort < φ s Trend Decision 1 Up Yes N/A Up Yes N/A Up Nothing 2 Up Yes N/A Up No N/A Flat Sell 3 Up Yes N/A Down N/A Yes Down Sell 4 Up Yes N/A Down N/A No Flat Nothing 5 Up No N/A Up Yes N/A Flat Nothing 6 Up No N/A Up No N/A Flat Nothing 7 Up No N/A Down N/A Yes Down Sell 8 Up No N/A Down N/A No Flat Sell 9 Down N/A Yes Up Yes N/A Flat Buy 10 Down N/A Yes Up No N/A Flat Buy 11 Down N/A Yes Down N/A Yes Down Nothing 12 Down N/A Yes Down N/A No Flat Nothing 13 Down N/A No Up Yes N/A Up Buy 14 Down N/A No Up No N/A Flat Nothing 15 Down N/A No Down N/A Yes Down Nothing 16 Down N/A No Down N/A No Flat Buy Policy 3 Forward Hedging This policy will utilize the forward market and its ability to hedge the positions taken on in the spot market. Participating in transactions in the forward market will protect from unexpected price fluctuations in the future. For example, if a decision to inject natural gas is made, the trader is anticipating a rise in prices in 72

83 CHAPTER 5: Model and Policies the future. However, this leaves the trader exposed to severe losses if prices drop. To avoid this exposure, the trader could decide to lock in the future price of natural gas by selling on the forward market. If the trader sells an amount of natural gas on the forward market equivalent to what was purchased on the spot, then there is guaranteed profit for the trader even if the price of natural gas on the spot market falls. The downside to this strategy is that covering the spot position 1:1 with a forward transaction eliminates the chance for large gains by an upwards movement of the spot price. Thus, a strategy would be to balance the risk and reward of the spot position by hedging only a certain percentage of the spot position with a forward position. This policy takes the decision of the amount to inject or withdraw from the spot market and hedge a percentage of the position with a decision to inject or withdraw in the future using the forward market. In order to avoid the complexities of a high dimensional problem, the spot decision will be derived from the policy in 5.3.4, using the optimal parameters found and fixing them for the current policy. The variables for this policy are expressed by η = the percent of spot position to hedge using forward transaction ρ s = the percentage level of price increase for short term trend φ s = the percentage level of price decrease for short term trend ρ l = the percentage level of price increase for long term trend φ l = the percentage level of price decrease for long term trend 73

84 CHAPTER 5: Model and Policies where, the last four being the optimal parameters found for Policy 2 which will be fixed. As in prior policies, x d,d+1 is the decision made on day d for the amount of natural gas to be injected or withdrawn from the spot market at time d+1. This will be derived from the fixed parameters above in order to produce a policy for the forward decision expressed by y d,m,m+1 = η x d,d+1, where, y d,m,m+1 is the forward decision made on day d and month m for delivery in month m+1, and η is a percent that will be tuned in order to obtain the optimal hedging percentage. 74

85 CHAPTER 6 Results With the aggregation of the natural gas storage model and the policies containing finely tuned parameters courtesy of the KGNP algorithm, estimates of the value of natural gas storage can be obtained. This chapter will present the results of the KGNP algorithm and subsequently the value of storage associated with each of the policies in Chapter 5. We endeavor to validate our results by garnering responses from our policies to the exposure of varying sample paths. We also compare our results to the current industry value for storage provided by Enstor Operating Company, a natural gas storage services company owned by Iberdrola Renewables. 6.1 Model and Algorithm Set-Up The characteristics of the natural gas storage facility which was utilized as parameters within the storage model can be found within the storage contract in Table 7.

86 CHAPTER 6: Results Table 7 - Salt Cavern Storage Contract with Model Parameters Term 01/01/ /31/2010 Type Depth Maximum Working Gas Capacity Initial Working Gas Capacity Maximum Injection Rate Maximum Withdrawal Rate Salt Cavern 3500 ft 7.5 Bcf 0 Bcf.3 Bcf/day.5 Bcf/day Fuel Injection Loss 1.5% Maximum/Minimum Facility Pressure Compressor.85 PSI/foot.2 PSI/foot 25,000 horse power For all policies, no assumptions were made as to the performance of specific alternatives, and thus equivalent priors were utilized within the KGNP algorithm. The bandwidths needed for the KGNP algorithm reflect the range of alternatives used within the policies. Due to comparable ranges amongst the policies, we were able to apply the four bandwidths throughout without variation. Both the priors and bandwidths for the KGNP algorithm used to solve our storage valuation problem can be found in Table 8. Table 8 - KGNP Priors and Bandwidths (h) μ x 0 β x 0 β x ε

87 CHAPTER 6: Results 6.2 Policy 0: Results for Short-Trend Speculation Denoted as Policy 0 in Chapter 5, this policy capitalizes on the short-term price movements of natural gas to maximize operating profit of the storage facility. The function of the KGNP algorithm is to perform sequential measurements over the alternatives of the policy s parameters, and after a budgeted amount of measurements will produce parameters that will show the best performance from the policy. For this policy, the KGNP algorithm produces parameters for ρ, the percentage level of price increase needed to instigate withdrawal, and φ, the percentage level of price decrease needed to instigate injection, where ρ and φ (0%,10%). Before revealing the parameters chosen by the KGNP algorithm, we will take a graphical look at the KGNP algorithm measurement choices by juxtaposing the measurement estimates with a plot of the knowledge gradient. Figure 25 - Policy 0: Second Measurement 77

88 CHAPTER 6: Results Figure 25 shows the first two measurements chosen by the KGNP algorithm. The algorithm will strive to pick measurements that will result in the maximum knowledge obtained. Typical of knowledge gradient policies, KGNP has chosen to first measure at the extremities of the alternatives. The first measurement is at (ρ, φ) = (0%, 0%) where it results in a negative estimate for the value. Notice in the graph of the estimation that the point equating to alternative (0%, 0%) is not the only alternative to shift in value. In fact, a great deal of alternatives in proximity to the measured alternatives are estimated to have a negative value as well, despite having not been measured. This is due to the correlation between alternatives which are assessed within the KGNP algorithm, allowing us to make assumptions regarding the estimates other alternatives. The second measurement is taken at another extremity, (0%, 10%), yet with greater success, finding that this alternatives yields a large value for the measurement. Despite two different values received from the two measurements, the graph depicting the value of the knowledge gradient displays a similar account of the knowledge obtained regarding the alternatives. The estimates and knowledge gained with prior measurements is important when determining the next measurement. Figure 26 shows several subsequent measurements branching out from (0%, 10%) and finding equally successful estimates at nearby alternatives. Yet as shown by the knowledge gradient, the value of knowledge is still high for many of the alternatives, leading us to assume that the KGNP algorithm will procure measurements with regards to those alternatives in subsequent iterations. 78

89 CHAPTER 6: Results Figure 26 - Policy 0: 25 Measurements This is continued in Figure 27 as the surface begins to form from the measurements surveyed by the KGNP algorithm. Figure 27- Policy 0: 100 Measurements 79

90 CHAPTER 6: Results After 500 measurements, the KGNP estimated surface for the annual profit produced from the operations of the storage facility with parameters ρ and φ can be observed in Figure 28. Figure 28 - Policy 0: 500 Measurements The maximum value of the surface is what the KGNP has deemed the best estimate for the annual profit from storage operations, using the corresponding parameters are considered optimal for this policy. These values are summarized in Table 9. ρ (%) 1.0% φ (%) 5.5% Maximum Value ($) $15,578,000 Table 9 - Policy 0: Results As expressed in the objective function in Section 5.2.7, the value of storage is equal to the expected cash flows rendered by its operations. Thus, storage takes on the value of $15,578,000 80

91 CHAPTER 6: Results for Policy 0, where a 1% price increase in the spot price dictates the withdrawal and sale of natural gas, and a 5.5% decrease in price calls for the purchase and injection of natural gas. An interesting observation can be found in the chasm between ρ, the price increase required to withdraw and φ, the price decrease required for injection. These values indicate that a larger percentage of decrease in price must occur before injecting gas, compared to the percent increase required for withdrawal. This can be contributed to the significantly higher costs necessary for injection versus withdrawal, due to the cost of using the compressor as well as the 1.5% of fuel loss associated with the injection process. The result of this higher cost for injection is that greater incentive is required to purchase the natural gas from the market in order to account for the added costs. Thus, our policy waits for a larger drop in price before making the decision to inject. 6.3 Policy 1: Results for Long Trend Speculation As stated in Chapter 5, Policy 1 capitalizes on the long-term price movements of natural gas to maximize operating profit of the storage facility, looking at trends within four weeks of the current date. The parameters ρ and φ maintain their identity from the previous policy. From our results for Policy 0, we expect that the parameters in Policy 1 will show the same disparity between the price increase required to withdraw, and the price decrease required for injection. We also expect the values for ρ and φ to increase due to the larger range in price movement that can result over a four week period. Therefore, the KGNP algorithm will measure alternatives where ρ and φ (0%,15%). 81

92 CHAPTER 6: Results We show again the evolution of the estimated surface and knowledge gradient surface where Figure 29 shows the first measurement. Once again, the algorithm tested first at an extremity. This figure also clearly displays the effects of correlation in the surrounding alternatives many of which have adjusted estimates based merely on the first measurement. Figure 29 - Policy 1: First Measurement In Figure 30, the evolution of the surface is extended through more measurements, and clearly shows the varying degrees of correlation between the measurement seen in the middle of the surface and surrounding alternatives as their estimated values cascade down from the pinnacle of the measured alternative. 82

93 CHAPTER 6: Results Figure 30 - Policy 1: 100 Measurements After 500 measurements, the KGNP algorithm procures the estimated surface for the storage value which can be observed in Figure 31. Figure 31 - Policy 1: 500 Measurements 83

94 CHAPTER 6: Results Table 10 - Policy 1: Results ρ (%) 2.0% φ (%) 9.5% Maximum Value ($) $18,579,000 The maximum value of the surface and the corresponding parameters are expressed in Table 10. In every respect, our expectations have been met. There is still the discrepancy between ρ and φ, due to the larger cost for injection versus withdrawal; and both parameters received a boost from their values in Policy 0 in order to adjust to the larger fluctuations in price that occur during a four week interval. 6.4 Policy 2 Long and Short Trend Recalling from Section 5.3.4, in order to use both the long term and short term price information, this policy contains four tunable parameters expressing certain criteria from which a decision to inject, withdraw, or do nothing will be instated. ρ s = the percentage level of price increase for short term trend φ s = the percentage level of price decrease for short term trend ρ l = the percentage level of price increase for long term trend φ l = the percentage level of price decrease for long term trend However, unlike the technique used in Policy 0 and Policy 1, these four parameters will not be tuned by the KGNP algorithm simultaneously due to dimensionality constraints. Thus, two of the parameters will be held constant as the KGNP algorithm estimates the additional parameters. The KGNP algorithm will then be run again with the fixed parameters now being the estimates from the previous 84

95 CHAPTER 6: Results KGNP process. This exchange between the parameters acting as fixed variables and being tuned by the KGNP algorithm will continue until the value for storage appears to converge. Table 11 describes the exchange in tuning the parameters, where the highlighted parameters are the ones tuned by the KGNP algorithm. Table 11 - Policy 2: Results KGNP Run ρ s φ s ρ l φ l Max Value 1 1% 5% 7.5% 7.5% $13,953, % 9% 7.5% 7.5% $14,722, % 9% 6.5% 13% $18,502,000 Optimal Estimate 2.5% 9.5% 1% 14.5% $18,742, Policy 3 Forward Hedge Policy 3 determines the appropriate percentage of the spot decision to hedge using a purchase or sale within the forward market. We attempt to hedge decisions from Policy 2 by holding the optimal parameters found in Policy 2 constant as the KGNP algorithm searches over the percentages. Unlike the technique in Policy 2, an exchange in the fixed variables does not occur since we are attempting to acquire the specific hedging strategy for a particular policy. We expect the value obtained for this policy to be less than that of Policy 2 since reward is being given up in exchange for protection from risk. 85

96 CHAPTER 6: Results Table 12 Policy 3 Parameters and results ρ s 2.5% φ s 9.5% ρ l 1% φ l 14.5% Percent Hedged 60% Max Value $6,120, Policy Analysis This section we will analyze our policies using fluctuating variables embedded within our model, such as the withdrawal rate and market volatility. We will also compare the results of our policies with actual data from recently traded storage volumes Withdrawal Rate Adjustment With our optimal parameters determined in 6.2 and 6.3, Policy 0 and Policy 1 were simulated one hundred times each, increasing the maximum withdrawal rate.01 Bcf/day every iteration. Therefore, we were able to achieve storage values for each policy with withdrawal rates ranging from 0 to 1 Bcf/day. The results are graphically represented in Figure 32. From the figure, we can see a deficiency in the value of storage when the withdrawal rate is between 0 and.05 Bcfs/day for the two policies. This is due to the fact that at these low deliverability levels, only an insignificant amount of gas has the ability to be withdrawn in order to capitalize on market movements. To gain some perspective on these rates, we establish that.05 Bcf/day is equivalent 50 MMcf/day, (1000 MMcf = 1 Bcf), a deliverability rate that is common amongst 86

97 Storage Value ($) CHAPTER 6: Results depleted reservoir storage facilities. As mentioned in Chapter 1, because of their low deliverability, depleted reservoir systems are used for seasonal cycling, which slowly injects natural gas during the spring when demand is low, and steadily withdraws it in the winter to meet heavy demand. Thus, this graph highlights the actual limitations of low deliverability storage in roles that require rapid operational maneuvering such as peaking services or market speculation. Figure 32 - Withdrawal Rate's Effect on Storage Value 2.50E E E E E E E E E Policy 0 Policy E E+07 Withdrawal Rate Bcf/Day Volatility Fluctuation The natural gas market is one of the most volatile markets, and thus to ensure that our policies would be stable when they encounter the volatility actually within the market, we have decided to create a heightened reality by augmenting the volatility within the pricing process two-fold. In Table 13, the numerical results 87

98 CHAPTER 6: Results can be found for Policies 0, 1, and 2, relating the value of storage with the standard volatility and then with the doubled volatility. Table 13 - Volatility Fluctuation Results Policy ς t 2*ς t Policy 0 $15,578,000 $3,164,145 Policy 1 $18,579,000-12,400,000 Policy 2 $18,742,000 $9,210,000 The results show a significant decrease in value upon the augmentation of the volatility. However, it should be noted that two of the three policies retain positive values despite the surge in volatility. Figure 33 displays the value surface of Policy 0 computed by the measurements of the KGNP. Recalling the surface in Figure 28 for the standard volatility, it can be seen that a shift in volatility will also shift the values of the policies, though not significantly enough to remove all value. However, this is just the case for Policy 0, seeing as the shift in volatility has greatly impacted the value of storage in Policy 1, implying that longer trend windows are more susceptible to swings in volatility. Figure 33- Policy 0, KGNP Result for Doubled Volatility 88