Stochastic Optimization of Power Systems Planning and Operations in High Penetration Renewables Scenarios Dr. Jean-Paul Watson Sandia National Laboratories Sandia National Laboratories is a multi-program laboratory managed and operated by Sandia Corporation, a wholly owned subsidiary of Lockheed Martin Corporation, for the U.S. Department of Energy s National Nuclear Security Administration under contract DE-AC04-94AL85000.
Talk Goals 1. To convey the conceptual basics of stochastic optimization 2. To convince you that stochastic optimization is pervasive in power systems operations and planning 3. To convince you that significant recent and upcoming algorithmic advances have made stochastic optimization practical in practice 4. To provide a brief survey of some research projects presently being conduced to support advanced algorithms for core power systems problems
Talk Goals All in 20 (now probably closer to 18) minutes!
Deterministic vs. Stochastic Optimization Deterministic Mixed-Integer Programming (MIP) The workhorse of (rigorous) Operations Research min Approximable for most real-world problems (NP-Hard) Stochastic Mixed-Integer Programming (SMIP) SMIP = MIP + uncertainty + recourse Still NP-Hard, but far more difficult than MIP in practice
On Risk Measures With stochastic optimization, there is a distribution of costs and the associated need to select a risk measure... Cost
Optimization and Power Systems Assertion # 1 Almost all power systems operations and planning decision problems with 5 minutes or greater look-ahead are mixed-integer optimization problems Support: MIP solvers are used widely by ISOs, every day Support: Scan IEEE TPS over the last decade Assertion # 2 All of these problems are really stochastic, but are not treated that way because of modeling and scalability issues Support: Scheduling with renewables is inherently uncertain Support: The uncertainty is there even without renewables! Support: Future budgets and costs are uncertain in expansion problems
So Why Isn t Stochastic Optimization Deployed in Power Systems Contexts? Modeling is significantly more complex Stochastic process models, multi-stage decisions Need significant expertise in both optimization and statistics This is part of the issue, but only part The real reason is that stochastic optimization problems are exceptionally difficult to solve But: Run-times are far from those required for operations, and don t really approach the turn-around times required for planning Advances in decomposition algorithmsover the past two decades, coupled with parallel computing, have changed the landscape!
The Impact of Decomposition: Biofuel Infrastructure and Logistics Planning Example of Progressive Hedging Impact: Extensive form solve time: >20K seconds PH solve time: 2K seconds Slide courtesy of Professor YueYue Fan (UC Davis)
The Impact of Decomposition: Wind Farm Network Design Where to site new wind farms and transmission lines in a geographically distributed region to satisfy projected demands at minimal cost? Formulated as a two-stage stochastic mixed-integer program First stage decisions: Siting, generator/line counts Second stage decisions : Flow balance, line loss, generator levels 8760 scenarios representing coincident hourly wind speed, demand Solve with Benders: Standard and Accelerated Slide courtesy of Dr. Richard Chen (Sandia California)
Next up Active power systems projects involving stochastic optimization Ranging from very researchy to very applied
Illustrative Project #1 Sponsored by the DOE Office of Science Program: Advanced Scientific Computing Research (ASCR) Important: ASCR has supported the development of algorithms and enabling mathematics for solving large-scale stochastic optimization problems for over 9 years
Multifaceted Mathematics for Complex Energy Systems Project Director: Mihai Anitescu, Argonne National Lab Goals: By taking a holistic view, develop deep mathematical understanding and effective algorithms to remove current bottlenecks in analysis, simulation, and optimization of complex energy systems. Address the mathematical and computational complexities of analyzing, designing, planning, maintaining, and operating the nation's electrical energy systems and related infrastructure. Multifaceted Mathematics for Complex Energy Systems Representative decision-making activities and their time scales in electric power systems. Image courtesy of Chris de Marco (U-Wisconsin). Integrated Novel Mathematics Research: Predictive modelingthat accounts for uncertainty and errors Mathematics of decisions that allow hierarchical, data-driven and real-time decision making Scalable algorithms for optimization and dynamic simulation Integrative frameworks leveraging model reduction and multiscale analysis Long-Term DOE Impact: Development of new mathematics at the intersection of multiple mathematical subdomains Addresses a broad class of applications for complex energy systems, such as : Planning for power grid and related infrastructure Analysis and design for renewable energy integration Team: Argonne National Lab (Lead), Pacific Northwest National Lab, Sandia National Labs, University of Wisconsin, University of Chicago
Multifaceted Mathematics for Complex Energy Systems (M2ACS) Project Director: Mihai Anitescu, Argonne National Lab The M2ACS missionis to develop deep mathematical understanding and effective algorithms to remove current bottlenecks in analysis, simulation, and optimization by taking a holistic view of the complex energy systems. $3.5M/year for 5 years M2ACS Team (22 researchers plus 9 post-docs & 10 GRAs) : Argonne National Laboratory (Lead) M. Anitescu (site lead), E. Constantinescu, P. Hovland, S. Leyffer, T. Munson, B. Smith, V. Zavala plus 4 post-docs and 3 GRAs Pacific Northwest National Laboratory H. Huang (site lead), M. Halappanavar, B. Lee, G. Lin, S. Lu, A. Tartakovsky plus 2 post-docs Sandia National Laboratories J.P. Watson (site lead), J. Siirola, plus 1 post-doc University of Chicago J. Birge (site lead), J. Weare plus 1 GRA University of Wisconsin M. Ferris (site lead), C. DeMarco, B. Lesieutre, J. Linderoth, J. Luedtke, S. Wright plus 2 post-docs and 6 GRAs
Illustrative Project #2 Sponsored by the DOE ARPA-e Program: Green Energy Network Integration (GENI) Note: An exemplar of ASCR-funded algorithmic research impacting an applied DOE office mission
Advanced Stochastic Optimization for Improved Power Systems Operations Sandia National Laboratories is a multi-program laboratory managed and operated by Sandia Corporation, a wholly owned subsidiary of Lockheed Martin Corporation, for the U.S. Department of Energy s National Nuclear Security Administration under contract DE-AC04-94AL85000.
Project Goals Execute stochasticunit commitment (UC) at scale, on real-world data sets Stochastic UC state-of-the-art is very limited (tens to low hundreds of units) Our solution must ultimately be useable by an ISO Produce solutions in tractable run-times, with error bounds Parallel scenario-based decomposition For both upper and lower bounding (Progressive Hedging and Dual Decomp.) Quantification of uncertainty Rigorous confidence intervals on solution cost Employ high-accuracy stochastic process models Leveraged to achieve computational tractability while maintaining solution quality and robustness Demonstrate cost savings on an ISO-scale system at high renewables penetration levels
The General Structure of a Stochastic Unit Commitment Optimization Model p 1 Objective: Minimize expected cost Generator Number 0 2 4 6 8 10 12 14 16 0 5 10 15 20 25 Hour of day p 2 p N First stage variables: Unit On / Off Nature resolves uncertainty Renewables output Forced outages Scenario 1 Scenario 2 Scenario N Second stage variables (per time period): Generation levels Power flows Voltage angles
Uncertainty in DAM, RUC, and SCED2 Stochastic Programming Models Reliability Unit Commitment Renewables generator output, load, forced (unplanned) outages Fewer binaries than DAM, long time horizon, many scenarios Look-Ahead Unit Commitment Similar to Reliability Unit Commitment Fewer binaries than RUC, short time horizon, few scenarios Day-Ahead Unit Commitment In contrast to RUC and SCED2, an ISO can t really make direct use of a stochastic UC in the DAM without changing DAM procedures With our partners, we are exploring alternative models and experimenting with procedures that incorporate stochastic models We are eager to discuss ideas offline
Impact of Scenarios on Decisions Too narrow Optimization fails to account for actual risks Too few low-cost units committed Cost: Start up additional highcost units Reliability: Shed load Must include a sample of high-impact, low-probability events Too wide Optimization result is too riskaverse Too many low-cost units committed Cost: Excessive no-load cost of committed units Environmental: spill variable generation No better than existing ruleof-thumb reserve rules
Scenario-Based Decomposition via Progressive Hedging (PH) Rockafellar and Wets (1991)
Illustrative Results: WECC-240 Test instance Modified WECC-240 instance for reliability unit commitment Stochastic demand, 100 scenarios Extensive Form (EF) CPLEX, after 1 day of CPU on a 16-core workstation No feasible incumbent solution PH, 20 iterations, post-ef solve - serial ~14 hours, 2.5% optimality gap PH, 20 iterations, post-ef solve parallel ~15 minutes, 2.5% optimality gap PH, 20 iterations, post-ef solve parallel with bundling ~15 minutes, 1.5% optimality gap
Illustrative Project #3 Sponsored by the DOE EERE Program: SunShot Note: An exemplar of ASCR-funded algorithmic research impacting an applied DOE office mission
Advancing Solar Deployment on Transmission Systems Abraham Ellis (PI), SNL
Open Tools for Operational Analysis Scope Develop, document and disseminate new, open-source tool that uses mixed-integer stochastic optimization techniques to study integration cost and system reliability for projected high solar deployment scenarios Approach Customize SUC modules based on existing optimization software, with interfaces to facilitate efficient data entry and parametric analyses Validate tool functionality on a reference case Make toolkit (including documentation and case studies) available through an open-source distribution model Impact By definition, high pen PV means higher uncertainty, more stark costreliability tradeoff. Stochastic tools are best suited to fully assess impacts. Access to a new tool that allows for more rigorous analysis of costs and reliability implications associated with high penetration solar scenarios
A few words on scenario selection
Scenario Sampling: How Many is Enough? Discretization of the scenario space is standard in stochastic programming Often, no mention of solution or objective stability Let alone rigorous statistical hypothesis-testing of stability Don t trust anyone who doesn t show you a confidence interval Various approaches / alternatives in the literature We like the Multiple Replication Procedure (MRP) introduced by Mak, Morton, and Wood (1999) Formal question we are concerned with x ) What is the probability that s objective function value is suboptimal by more than α%? But making due with a fixed set or universe or scenarios
The Multiple Replication Procedure (Mak, Morton, Wood 1999) Slide 27 From Bayraksan and Morton (2009) Assessing Solution Quality in Stochastic Programs Via Sampling
Illustrative MRP Results: Wind Farm Network Design 1000 scenarios, randomly sampled from a universe of 8760 scenarios Key results: Objective function value is remarkably stable across different parameterizations of the procedure Confidence interval widths are relatively small for a planning problem Results are stable across replications of the same parameterization of the MRP procedure Practical impact: We don t need 8760 scenarios! Looking to stochastic UC: We need more scenarios, not less
A Word on Enabling Technology
Our Software Environment: Coopr Project homepage http://software.sandia.gov/coopr The Book Mathematical Programming Computation papers Pyomo: Modeling and Solving Mathematical Programs in Python (Vol. 3, No. 3, 2011) PySP: Modeling and Solving Stochastic Programs in Python (Vol. 4, No. 2, 2012)
Talk Goals, Revisited 1. To convey the conceptual basics of stochastic optimization 2. To convince you that stochastic optimization is pervasive in power systems operations and planning 3. To convince you that significant recent and upcoming algorithmic advances have made stochastic optimization practical in practice 4. To provide a brief survey of some research projects presently being conduced to support advanced algorithms for core power systems problems
QUESTIONS JWATSON@SANDIA.GOV