The Coming Big Data Tsunami in Energy Market Analytics. Jeff Fong, Co-Founder, F S DataScience October 2014

Transcription

1 The Coming Big Data Tsunami in Energy Market Analytics Jeff Fong, Co-Founder, F S DataScience October 2014

2 The views expressed in the following material are the author s and do not necessarily represent the views of the Global Association of Risk Professionals (GARP), its Membership or its Management. 1

3 Outline I. Why are Big Data & Analytic concepts important in Energy Markets? II. What is Big Data & Analytics? III. A small Big Data & Analytics example IV. Big Data, Analytics and Energy Markets V. Getting started if you haven t already

4 Why are Big Data & Analytic concepts important in Energy Markets?

5 Big Data & Analytics The Past: A competitive advantage was often obtained with an asset. Railroads, pipelines, refineries

6 Big Data & Analytics The Past: A competitive advantage was often obtained with an asset. Railroads, pipelines, refineries

7 Big Data & Analytics The Past: A competitive advantage was often obtained with an asset. Railroads, pipelines, refineries, Water utilities Power generation

8 Big Data & Analytics The Past: A competitive advantage was often obtained with an asset. Railroads, pipelines, refineries, Water utilities Power generation Asset based competitive advantage Monopolies

9 Big Data & Analytics What we see today is a trend where analytics revolutionize industries Lower capital intensive industries were first We have already seen many industries revolutionized by companies that specialize in data and analytics (Google, Amazon) Advertising and Sales Logistics Travel and Airlines

10 Big Data & Analytics What we see today is a trend where analytics revolutionize industries Lower capital intensive industries were first We have already seen many industries revolutionized by companies that specialize in data and analytics (Google, Amazon) Advertising and Sales Logistics Travel and Airlines Energy markets are next! I. The data is being stored II. Sufficient processing power is now available III. Markets are becoming integrated

11 Big Data & Analytics As energy markets continue to become more global, the competitive advantage from analytics will continue to grow increased integration of global energy markets new pipelines, transmission, energy technologies growing amounts of data increased competition and demand response integration with financial markets The future of strategic discussions will include include data analytics and algorithms as being competitive advantages

12 What is Big Data & Analytics?

13 What is Big Data & Analytics Big is relative data starts to become big when you can t make use of the raw data without summarizing it Three types of analytics 1. Descriptive 2. Predictive 3. Prescriptive

14 Analytics: Descriptive Most data in its raw form is not suitable for human consumption Data must be condensed into useful pieces of information The purpose of descriptive analytics is to summarize what has happened The focus is on learning about your data (data discovery) 80% of business analytics is descriptive (think about your BI reports)

15 Analytics: Descriptive Averages, variance, correlations, principle components, distributions and summary statistics are all descriptive They help you learn about your data, but are not in themselves, predictive or prescriptive Example: How do interest rates, spot prices and volatility move together?

16 Analytics: Predictive Predictive analytics do not tell you will happen; it tells you what might happen. (probabilistic) In other words, using data you do have; fill in data you don't have predict possible future data fill in possible values for missing data

17 Analytics: Predictive (=modeling) Predictive analytics do not tell you what will happen; it tells you what might happen. (probabilistic) In other words, using data you do have; fill in data you don't have predict possible future data fill in possible values for missing data Model building is a big part of predictive analytics Statistical models, data mining, machine learning, time-series, monte-carlo simulation, forecasting, neural nets are all predictive They tell you what might happen, but not what to do about it

18 Analytics: Predictive Example: Predict or model the value of an OTC call option We use data we have (interest rates, spot prices and volatility) to predict data we don t have (the call option value) VXX SPY TLT

19 Analytics: Predictive Example: Predict or model the value of an OTC call option We use data we have (interest rates, spot prices and volatility) to predict data we don t have Possible future interest rates, spot prices and volatility And possible future option values VXX SPY TLT CALL Option

20 Analytics: Prescriptive Using our data, descriptive analytics and predictive models We can prescribe an action that best achieves an objective Prescriptive analytics needs actionable data What actions can be taken that affect outcomes? What are the objectives? VXX SPY Minimize risk? Maximize reward? Maximize Sharpe ratio? TLT CALL Option

21 Analytics: Prescriptive Example: Actions to hedge a call option We have the data for interest rates, spot prices & volatility We have a predictive model for an option price Objective: Maximize return per unit of risk VXX Dynamic hedging is a prescriptive strategy delta / gamma hedge hedge greek exposures TLT SPY CALL Option

22 A small Big Data & Analytics example

23 A Portfolio Trading example 1. SPY - S&P500 index 2. TLT year Treasury bonds ETF 3. VXX S&P500 Volatility ETF VXX SPY TLT

24 A Portfolio Trading example Mini universe of 9 stocks and ETFs 1. SPY - S&P500 index 2. TLT year Treasury bonds ETF 3. VXX S&P500 Volatility ETF 4. JNK - High yield corporate bond ETF 5. UGA - Gasoline futures ETF 6. VLO - Valero Energy Corp (refinery) 7. USO - Oil (WTI) futures ETF 8. JO - Coffee ETF 9. SBUX - Starbucks VXX TLT SPY

25 A Portfolio Trading example Mini universe of 9 stocks and ETFs 1. SPY - S&P500 index 2. TLT year Treasury bonds ETF 3. VXX S&P500 Volatility ETF 4. JNK - High yield corporate bond ETF 5. UGA - Gasoline futures ETF 6. VLO - Valero Energy Corp (refinery) 7. USO - Oil (WTI) futures ETF 8. JO - Coffee ETF 9. SBUX - Starbucks Complex dynamics VXX TLT JNK SPY USO JO SBUX VLO UGA How can we methodically construct a trading strategy?

26 Descriptive Mini universe of 8 stocks and ETFs Calculate: o o o means, variances correlations, autocorrelations principle components Discover features about this dataset

27 Predictive This small example isn t really that small. There are 2^8=256 possible combinations of underliers to include, each with endless possibilities of share combinations With some data mining (We ll make the code available for you to try yourself), we can build a (predictive) model of a cointegrated portfolio from our mini universe of stocks and ETFs

28 Predictive This small example isn t really that small. There are 2^8=256 possible combinations of underliers to include, each with endless possibilities of share combinations With some data mining (We ll make the code available for you to try yourself), we can build a (predictive) model of a cointegrated portfolio from our mini universe of stocks and ETFs In otherwords, we can build a portfolio that mean reverts using predictive (modeling) analytics

29 Predictive Mini universe

30 Predictive Mini universe Data-Mined subset

31 Predictive Mini universe Data-Mined subset Cointegrated portfolio allocation 1. long shares of $40.31 = $ short shares of $48.87 = ($101.53) 3. short shares of $48.36 = ($58.08) 4. short shares of $30.64 = ($83.80)

32 Predictive 5 year history of the net value of the model cointegrated portfolio

33 Predictive Mean Reversion Level 5 year history of the net value of the model cointegrated portfolio Distribution of the net portfolio value over the 5 years

34 Predictive Mean Reversion Level Let s create a simple (prescriptive) strategy with the objective being to generate trading profits! This is not a recommendation! Do your own due diligence!

35 Prescriptive BUY the portfolio when it s at the 5 th percentile Let s create a simple (prescriptive) strategy with the objective being to generate trading profits! This is not a recommendation! Do your own due diligence!

36 Prescriptive BUY the portfolio when it s at the 5 th percentile SELL the portfolio when it s at the 95 th percentile Let s create a simple (prescriptive) strategy with the objective being to generate trading profits! This is not a recommendation! Do your own due diligence!

37 Prescriptive Cover at the Mean Reversion Level BUY the portfolio when it s at the 5 th percentile SELL the portfolio when it s at the 95 th percentile Let s create a simple (prescriptive) strategy with the objective being to generate trading profits! This is not a recommendation! Do your own due diligence!

38 Prescriptive Cover at the Mean Reversion Level Come back and ask me what happened in 6 months! BUY the portfolio when it s at the 5 th percentile SELL the portfolio when it s at the 95 th percentile Let s create a simple (prescriptive) strategy with the objective being to generate trading profits! This is not a recommendation! Do your own due diligence!

39 Big Data, Analytics and Energy Markets

40 Energy Market Big Data Analytics Modeling energy markets is a difficult and complex problem complex distributions large constrained optimization problems market feedback loops Problems in energy markets, get very big, very fast!

41 Energy Market Big Data Analytics Digging into the details is like opening pandora s box If we look at a power plant, at its surface it seems simple Power Power Plant Gas Basis

42 Energy Market Big Data Analytics Digging into the details is like opening pandora s box If we look at a power plant, at its surface it seems simple Load Temp EIA Power Power Plant WTI Gas Heating Oil Gas Basis Once we peel back the surface, it gets complicated very fast

43 Energy Market Big Data Analytics Digging into the details is like opening pandora s box If we look at a power plant, at its surface it seems simple EIA How sensitive is my portfolio to EIA numbers? What is the optimal hedge? Gas Power Gas Basis Load Power Plant Temp Heating Oil WTI What other assets complement my existing portfolio What s the impact of a transmission or pipeline disruption? Once we peel back the surface, it gets complicated very fast

44 The Real World

45 The Real World Hundreds of nodes and relationships Dozens of different asset types Many constraints and market rules to consider A lot of analytics and software/hardware are required to analyze such complex problems

46 Energy Market Big Data Analytics Even the largest problems can still be broken down Use an analytics based approach to simplify the problems Include top down quantitative methods with bottoms up fundamental based modeling Leverage Big Data technology once its needed Hadoop (reading / writing) Distributed and cloud computing (large scale computations) Hadoop and distributed / cloud technologies can be extremely powerful in helping you solve large problems

47 Getting started if you haven t already

48 Learn as much as you can! Learn to work with and analyze large data sets SQL (any will do) Any good programming or statistical language such as R

49 Learn more! Learn to work with and analyze large data sets SQL (any will do) Any good programming or statistical language such as R Start by solving small optimization problems to gain experience Free online universities and excellent courses udacity, coursera, etc Learn more about other fields where you have interests

50 Learn more! Learn to work with and analyze large data sets SQL (any will do) Any good programming or statistical language such as R Start by solving small optimization problems to gain experience Free online universities and excellent courses udacity, coursera, etc Learn more about other fields where you have interests Go write the ERP exam!

51 Gaining Adoption Gaining adoption can be difficult at first Start with small, easy to handle projects Try descriptive analytics first Focus on solving specific problems first to show ROI Share results often and invite feedback Incorporate feed back and use rapid prototyping Teach & collaborate!

52 Resources Free online courses coursera.org udacity.com Open source software options hadoop.apache.org Javascript libraries Commercial software options Matlab MS SQL / Oracle

53 QUESTIONS?

54 C r e a t i n g a c u l t u r e o f r i s k a w a r e n e s s Global Association of Risk Professionals 111 Town Square Place 14th Floor Jersey City, New Jersey U.S.A nd Floor Bengal Wing 9A Devonshire Square London, EC2M 4YN U.K (0) About GARP The Global Association of Risk Professionals (GARP) is a not-for-profit global membership organization dedicated to preparing professionals and organizations to make better informed risk decisions. Membership represents over 150,000 risk management practitioners and researchers from banks, investment management firms, government agencies, academic institutions, and corporations from more than 195 countries and territories. GARP administers the Financial Risk Manager (FRM ) and the Energy Risk Professional (ERP ) Exams; certifications recognized by risk professionals worldwide. GARP also helps advance the role of risk management via comprehensive professional education and training for professionals of all levels Global Association of Risk Professionals. All rights reserved.