Active network management for electrical distribution systems: problem formulation and benchmark

Transcription

1 Active network management for electrical distribution systems: problem formulation and benchmark Q. Gemine D. Ernst B. Cornélusse Department Electrical Engineering and Computer Science University Liège 04 Dutch-Belgian RL Workshop Brussels Belgium

2 Motivations Environmental concerns are driving growth renewable electricity generation Installation wind and solar power generation resources at distribution level Current fit-and-forget doctrine for planning and operating distribution network comes at continuously increasing network reinforcement costs GREDOR - Gestion des Réseaux Electriques de Distribution Ouverts aux Renouvelables

3 Active Network Management ANM strategies rely on short-term policies that control power injected by generators and/or taken f by loads so as to avoid congestions or voltage problems. Simple strategy: Curtail production generators. More advanced strategy: Move consumption loads to relevant time periods. Such advanced strategies imply solving largescale optimal sequential decision-making problems under uncertainty. GREDOR - Gestion des Réseaux Electriques de Distribution Ouverts aux Renouvelables 3

4 Observations Several researchers tackled this operational planning problem. They rely on different formulations problem making it harder for one researcher to build on top anor one s work. We are looking to provide a generic formulation problem and a testbed in order to promote development computational techniques. GREDOR - Gestion des Réseaux Electriques de Distribution Ouverts aux Renouvelables 4

5 Problem description We consider problem faced by a DSO willing to plan operation its network over time while ensuring that operational constraints its infrastructure are not violated. This amounts to determine over time optimal operation a set D electrical devices We describe evolution system by a discrete-time process having a time horizon T (fast dynamics is neglected). GREDOR - Gestion des Réseaux Electriques de Distribution Ouverts aux Renouvelables 5

6 P (MW) 5 4 Control Actions 3 Control actions are aimed to directly impact power levels devices d D. Figure 3: Curtailment a distributed generator. 0 Time 8 Potential prod. Modulated prod. 7 8 Potential prod. Modulated prod. We also consider that DSO can modify consumption flexible loads. These loads constitute a subset F out whole set loads C D network. An activation fee is associated to this control mean and flexible loads can be notified activation up to time immediately preceding start service. Once activation is performed at time t0 consumption flexible load d is modified by a certain value during Td periods. For each se modulation periods t [[t0 + ; t0 + Td ]] this value is defined by modulation function Pd (t t0 ). An example modulation function and its influence over consumption curve curt is presented in Figure 4. Curtailment instructions can be imposed to generators. e Cost: En [MWh] Price MWh 6 7 P (MW) P (MW) 6 0 Time Standard cons. Figure 3: Curtailment a distributed generator. Flexibility service loads can We also consider that DSO can modify consumption flexible loads. a subset F out whole set loads C generator. D network. An ac Figure 3: Curtailment a distributed beconstitute activated. Modulated cons. 8 0 E + 6 is associated to this control mean and flexible loads can be notified activation up immediately preceding start service. Once activation is performed at 4 We also consider that flexible DSO can consumption flexible loads. consumption load modify d is modified by a certain value during Td periods. 3 se modulation t [[t0 set + ; t0 + Tdloads ]] this Cvalue defined by modulati constitute a subset F out periods whole Dis network. An ac Pdto (t this t0 ).control An example modulation and influence consum is associated meanand flexible function loads can beitsnotified over activation up is presented in Figure 4. service. Once activation is performed at immediately preceding start P (k W ) P d (k W ) Time Cost: activation fee E 0.5 Time-coupling effect. consumption flexible load d ist modified by a certain value during Td periods. +T t T ime se modulation periods t [[t + ; tover Td ]]conthis value is defined by modulati GREDOR - Gestion Electriques aux Renouvelables 6 0signal 0 + (a) Modulation signal des Réseaux consumption (Td = 9). de Distribution (b) Impact Ouverts modulation Pd (t sumption. t0 ). An example modulation function and its influence over consum is presented in Figure 4. E t t 0 (time) f

7 Problem Formulation The problem computing right control actions is formalized as an optimal sequential decision-making problem. We model this problem as a first-order Markov decision process with mixed integer and continuous sets states and actions. s t+ = f(s t a t w t+ ) s t s t+ S a t A st w t+ p( s t ) GREDOR - Gestion des Réseaux Electriques de Distribution Ouverts aux Renouvelables 7

8 System state st in st The electrical quantities can be deduced from power injections devices. Active power injections loads and power level primary energy sources DG (i.e. wind and sun). The control instructions DSO that affect current period and/or future periods are also stored in state vector. in st Upper limits on production levels and number active periods left for flexibility services s t =(P t...p C t ir t v t P t...p G t s (f) t...s(f) F t q t) GREDOR - Gestion des Réseaux Electriques de Distribution Ouverts aux Renouvelables 8

9 hus governed by relation fmhour in day qt allows capturing daily strends v (m/s) process. Given hypos evolution from a state st to astate t+ is described by transition f onstant power factor for devices reactive power consumption can directly be ded state st+ depends in saddition state control action = f (s at wtcurve )preceding () Figure 5: tto Power a wind generator. t+ rom Pdt+ : realization sstochastic processes modeled as Markov proces Q = tan P. qt+ probability d dt+ W and such that it follows a conditional law pw ( st ). In order to define y we havesolar irradiance and photovoltaic production on In in more detail now describe procedure various processes it. consumptio Section 5 wewe describe a possible to modelthat constitute evolution f : S A W 7! S Transition Function sible realizations af :random process. The general evo S A W! 7 S s n aggregated residential consumers generators using relation (3). Like wind set generators photovoltaic inherit ir uncertainty in production yadrelation from uncertainty associated to ir energy source. This source is represented by le consumption is set possible realizations a random process. The general evolu solar irradiance which ispower power level wind incident solar energy per m. The irradiance.3. Wind speed and level generators Set possible realizations Curtailment instructions for thus governed by relation tainty about behavior consumers inevitably leads to uncertainty about is stochastic process that we model while production level is obtained by a determi s = f (s a w ) next period and activation y withdraw from network. However over one day process horizon some trends a random with wcan iswsim t+ t t t t uncerta The uncertainty about production level wind turbines is panels. inherited from function irradiance and surface a photovoltaic This function d. Consumption peaks for example early and in evening for st+ =infand (sthat adefined wmorning ) flexible loads. bout wind speed. The Markov process that consider governs wind speed w t is twe t that power curvearise wind generators as follows a conditional at uniform levels that fluctuate from onethe day production to anor and among sconsumers assumed but to be over network. level consumers. wind generato probability law WpWspeed P = surf ir model evolution consumption each load d C by hen obtained by using a deterministic function that depends law wind gt g g t W and such that it follows a conditional probability ( sttrealization ). In orde unction power curve describe considered generator. Weprocesses can formalize this phenomenoit ionwhere in ifmore detail we now various that constitute g is efficiency factor constant and equal to 5% Pdt+ = fd (P (3)while t wdt )assumed dt qpanels and is specific (v) is surface panels in m vt+ = fv (vt qt wt ) to each photovoltaic generator. The irrad is alevel component by wtir pw ( s ). Thephenomenon dependencyis functions fd following to quarter is denoted twhole modeled by Markov pro t and oad consumption Pgt+ g (vt+ ) 8g windprocess. generators G hyposis day qt allows capturing =daily trends Given (ir) irt+ fir(irconsumers ) inevitably ower factor for devices =reactive can leads directlyto beuncertainty deduced t qpower t wt consumption rtainty about behavior (v) uch that wt is a component wt surf pw ( s where g is power curve gener t ) and : P = ir 8g solar generators G el y withdraw fromgt+ g g However t+ network. over a one day horizon some. A typical example power curve (v) is illustrated in Figure 5. Like loads produc g Qqt+ = tan d Pdt+. ed. Consumption peaks using: arise for example in early morning and in e (v) f reactive power such that wt isisobtained a component wt pw ( st ). The technique used in Section 5 to bui ion 5GREDOR we describe a at possible procedure to model evolution consumption 9 Gestion des Réseaux Electriques de Distribution Ouverts aux Renouvelables l consumers but levels that fluctuate from one day to anor and among c from a dataset is similar to one wind speed case. Qgt+ = tan(3). ted set residential consumers using relation g Pgt+. at it follows a conditional probability law p ( s ). In or l we now describe various processes that constitute on { e behavior consumers inevitably leads to uncertaint from network. However over a one day horizon som arise for example in early and in th wepeaks model evolution consumption each morning load d C by

10 0=h n (e fp nt Q nt ) (4) and in each link l L wehave: Reward Function 3.4 Reward function and goal i l (e f) =0. (5) The reward function r : S A s S 7! R associates an instantaneous rewards for each transition system from a period t to a period t+: In order to evaluate performance a policy we first specify reward function r : S A s S 7! R that associates an instantaneous rewards for each transition system from a period t to a period t + : r(s t a t s t+ )= X gg P gt+ max{0 P gt+ 4 }Cg curt (q t+ ) {z } curtailment cost DG X df a (f) dt Cflex d {z } activation cost (s t+ ) {z } barrier function flexible loads ]0; [ is discount factor. Given that t < for t>0 furr in tim n from period t = 0 less importance is given to associated reward. B Because operation a DN must be always be ation where Cg curt (q t+ ) is day-ahead market price pour quarter hour q t+ in day and ensured a DN must we be consider always be ensured return it does R not over seem an relevant infinite to consider C flex d is activation cost flexible loads specific to each m. The function is a barrier trajectory function that allows tosystem: penalize a policy leading system in a state that violates operational limits. It is defined as TX nite number periods and we introduce return R as R = R = lim (s t+ )= nn[ X (e nt+ T +! fnt+ V n)+ (V n e nt+ fnt+)] t=0 + X ll t r(s t a t s t+ ) (6) ( I lt+ I l ) (7) esponds GREDOR to - Gestion des weighted Réseaux Electriques sum de Distribution rewards Ouverts aux observed Renouvelables over an infinite trajectory 0 Given that costs and penalties have finite values and that reward fun

11 also observe that because s t+ = f(s t a t w t ) it exists a function : S A regates functions f and r and such that Optimal Policy r(s t a t s t+ )= (s t a t w t ). p W ( s t ). Let : S 7! A s be a policy that associates a control action to e Let We can : Sdefine 7! A starting be a policy from an that initial associates state s s 0 a = control s exp ystem. by We can define starting from an initial state s 0 = s expectedret action to each state system expected return cy by this policy can be written as: J (s) = lim T! TX EE { { w t p W ( s t ) w t p W ( s t ) t=0 t (s t (s t ) w t ) s 0 = s}. (s t a t w t )=r(s t a t f(s t a t w t )) yte by Let space be space all space policies policies all stationary.. For For a DSO a DSO policies. addressing addressing Addressing operational described operational in Section planning is equivalent problem to determine a andso optimal consists policy in amon cribed in Section is equivalent s finding i.e. a an policy optimal that satisfies policy following : condition to determine optimal policy J (s) J (s) 8s S 8. l know that such a policy satisfies Bellman equation [9] which can be wri J (s) = max GREDOR - Gestion des Réseaux w p Electriques de Distribution Ouverts aux Renouvelables W ( s) aa s r(s t a t s t+ )= (s t a t w t ) W( s t ). Let : S 7! A s be a policy that associates a control ac J (s) = lim T! E T X t=0 t (s t (s t ) w t ) s 0 = s}. i.e. a policy that satisfies following condition J (s) J (s) 8s S 8. ow that such a policy satisfies Bellman equation [9] which ca (s a w)+ J (f(s a w)) 8s S.

12 Solution Techniques We identified three classes solution techniques that could be applied to operational planning problem: mamatical programming and in particular multistage stochastic programming; approximate dynamic programming; simulation-based methods such as direct policy search or MCTS. GREDOR - Gestion des Réseaux Electriques de Distribution Ouverts aux Renouvelables

13 Test Instance We designed a benchmark ANM problem with goal promoting computational research in this complex field. Figure 6: Test network. on k parameter values i (i I (k) ) are evaluated by simulating trajectories system are associated to policies i. The result se simulation allows selection arameter values i (i I (k+) ) for next iteration. The goal such an algorithm is verge as fast as possible towards a parameter value ˆ that defines a good approximate l policy ˆ. or subset simulation-based methods is Monte-Carlo tree search technique [6 each time-step this class algorithms usually rely on simulation system trajecto build incrementally a scenario tree that does not have a uniform depth. These are evious simulations that are exploited to select nodes scenario tree that have to eloped. When construction tree is done action that is deemed optimal for ot node tree is applied to system. est instance The set models and parameters that are specific to this instance as well as documentation for ir usage are accessible as a Matlab class at 0 s section we describe a test instance considered problem. The set models arameters that are specific to this instance as well as documentation for ir usage cessible at as a Matlab r class. It has been ped to provided a black-box-type simulator which is quick to set up. The DN on which stance is based is a generic DN 75 buses [8] that has a radial topology it is presented re 6. We bound various electrical devices to network in such a way that it is possible 65 GREDOR - Gestion des Réseaux Electriques de Distribution Ouverts aux Renouvelables 3 her nodes this network into four distinct categories: X Pdt (MW) Pd(t ) (MW) d D d D Temps Figure 8: Power withdrawal scenarios devices. Negative values indicate t more power than what is consumed by loads. T ime ach residential node is connection point a load that represents a set residential 60

14 Example Policy In order to illustrate operational planning problem and test instance let s consider a simple solution technique. It consists in a simplified version a multi-stage stochastic program: Figure : Scenario tree that is built at each time step. GREDOR - Gestion des Réseaux Electriques de Distribution Ouverts aux Renouvelables 4

15 In order to operate an electrical distribution network in a secure and cost-e cient way it is necessary due to rise renewable energy-based distributed generation to develop Active Network Management (ANM) strategies. These strategies rely on short-term policies that control power injected by generators and/or taken o by loads in order to avoid congestion or voltage problems. While simple ANM strategies would curtail production generators more advanced ones would move consumption loads to relevant time periods to maximize potential renewable energy sources. However such advanced strategies imply solving large-scale optimal sequential decision-making problems under uncertainty something that is understandably complicated. In order to promote Example Policy In order to illustrate development computational operational techniques for active network planning management we problem detail a generic procedure for formulating ANM decision problems as Markov decision processes. and test Weinstance also Figure specify : it Scenario to a 75-bus tree let s that distribution built consider atnetwork. each timethe step. resulting a simple test instance is available solution at It can be used as a test bed for comparing technique. presented solution It existing technique consists computational Figurecan : bescenario written techniques as in tree that a is built simplified at each time step. version a 3 : as well as for developing new ones. A solution technique that consists in an approximate multistage program is also illustrated on test instance. X X multi-stage ˆ stochastic program: D (s) = arg min min hp k P gk k C curt presented solution technique can be written as 3 g (q : k )+ P Index terms aa s (s) Active 8kK t : s k network management electric 4 distribution gk kk t \{0} gg i 8kK t \{0}: a Ak X M gk + MX ˆ gk D (s) = arg min min hp + X network flexibility services h i renewable energy optimal sequential decision-making undercuncertainty k P flex gk k Cg curt d a (f) d0 (54) large system s.t. s df (q k )+ Pgk 0 = s (55) aa s (s) 8kK t : s k 4 kk t \{0} gg i 8kK t \{0}: Ak M gk + Mgk + X h i Introduction a 0 = a (56) C flex d a (f) s d0 (54) k = f(s Ak a Ak w Ak ) 8k K t \{0} (57) s.t. s df In Europe 0/0/0 objectives a Ak A 0 sak = s 8k European K t \{0} Commission and(58) consequent (55) financial incentives established by a (f) locala = a (56) A k = 0 governments 8k {k K t are D k currently > } driving (59) growth electricity generation from renewable energy s s k = f(s Ak a Ak w Ak ) 8k K t \{0} (57) k Ŝ sources []. A substantial part investments lies in (ok) 8k K t \{0} (60) distribution networks (DNs) and aconsists Ak A sak 8k installation K t \{0} units that depend on(58) wind or sun P gk = max(0p gk P gk ) 8(g k) G K t \{0} (6) as a primary energy source. The significant increase number se distributed generators (DGs) undermines fit and forget doctrine which has dominated planning (59) and M a (f) gk = max(0 A k = 0 P gk 8k P gk {k) 8(g K k) t D G k > K} t \{0} (6) whereoperation Equation (59) enforces DNs upthat to this activation point. s k This Ŝ flexible (ok) doctrine loads 8k was isk not t \{0} developed accounted when as a recourse DNs had (60) sole mission action. The delivering set Ŝ(ok) k is anenergy approximation coming from set P S gk = max(0p transmission (ok) system states that respect gk P network gk ) 8(g (TN) k) G to K t \{0} consumers. (6) With operational this approach limits. For adequate test instance investments presented in inthis network paper this components set is defined(i.e. usinglines a linear cables transformers constraint over upper limits active production M gk = levels max(0 and over P gk Pactive gk ) 8(g consumption k) G K t \{0} (6) etc.) must constantly be made to avoid congestion and voltage problems without requiring con- Equation monitoring (59) enforces and loads: wheretinuous n that control activation flexible loads is not accounted as a recourse action. The set Ŝ(ok) Ŝ (ok) s S X P k is an approximation g + X power flows orovoltages. To that end network planning is done with respect to a set critical scenarios P d + P set d consisting < S (ok) C production(63) and demand levels in system states that respect gg dc operational limits. For test instance presented in this paper this set is defined using a linear where constraint C is aover constant upper that can limits be estimated activeby production trial and error levels and and withover P dk defined activeasconsumption in Equation loads: (0). The physical motivation behind this constraint is that issues usually occur when GREDOR - Gestion a high des level Réseaux distributed Electriques production de n and Distribution a low Ouverts aux Renouvelables 5 Ŝ (ok) s S X consumption P g + X level take place simultaneously. o In order to get control actions that are somehow robust top d + evolutions P d < C system that (63) gg dc would not be well accounted in scenario tree objective function Problem (54)-(6) includes for each node k but root node following terms:

16 Example Policy 4: Example a simulation run system controlled by policy (54)-(6) ove GREDOR - Gestion des Réseaux Electriques de Distribution Ouverts aux Renouvelables 6

17 Thank you [en] GREDOR - Gestion des Réseaux Electriques de Distribution Ouverts aux Renouvelables 7

18 References [] Q. Gemine E. Karangelos D. Ernst and B. Cornélusse. Active network management: planning under uncertainty for exploiting load modulation. In Proceedings 03 IREP Symposium - Bulk Power System Dynamics and Control - IX page [] W.B. Powell. Clearing jungle stochastic optimization. Informs TutORials 04. [3] B. Defourny D. Ernst and L. Wehenkel. Multistage stochastic programming: A scenario tree based approach to planning under uncertainty chapter 6 page 5. Information Science Publishing Hershey PA 0. [4] L. Busoniu R. Babuska B. De Schutter and D. Ernst. Reinforcement Learning and Dynamic Programming Using Function Approximators. CRC Press Boca Raton FL 00. [5] Q. Gemine D. Ernst and B. Cornélusse. Active network management for electrical distribution systems: problem formulation and benchmark. Preprint arxiv Systems and Control 04. GREDOR - Gestion des Réseaux Electriques de Distribution Ouverts aux Renouvelables 8