NYISO DECISION SUPPORT SYSTEM INTERMEDIATE TRAINING COURSE Elisabeth Steele and Michele Goyette August 2004
2 Agenda Objectives Review of Introductory Topics Course Background Review of Introductory Topics Corporate Reports Conceptual Data Models Universe Design Overview Business Objects Features Overview Custom Querying Custom Report Formatting Automatic summation Additional Intermediate Features Web Intelligence
3 Objectives Reinforce Introduction Course Topics Overview of the Data Elements Contained in the DSS (Business Objects Universes) Query DSS for Real Life Business Scenarios Create Custom Reports Using Features of DSS Toolset WILL NOT INCLUDE: Advanced Business Objects training available from third parties.
Retrieve Corporate Reports Create and Format Custom Report Create Complex Reports 4
5 DSS Training Offerings Introduction Webex Basics Instructor Led Self-directed Introduction Planned NYISO Web Site Intermediate Classroom Creating Reports Instructor Led Third Party Offerings Independent Advanced Business Objects
6 Supporting Documentation NYISO DSS Users Guide DSS Release Matrix Lists of Interval Days and Invoice Versions in DSS DSS Customer Settlements Billing Code Mapping Document NYISO Billing and Accounting Manual
Review of Introductory Topics
8 Introduction Course Data Warehouse DSS Data Scope Functional Summary Versioning Strategy Data Availability Corporate Reports Summary / List of Reports How to Access
9 Data Warehouse Definition Data Warehouse Repository of integrated, non-volatile and timevariant data, collected from heterogeneous transaction processing applications, and stored in a format optimal for reporting and strategic analysis for an enterprise -- Bill Inmon For NYISO Data from BAS, MIS and Consolidated Invoice applications is collected, formatted and stored in a single, centralized location for customer reporting and analysis.
10 Data Warehouse Definition NYISO Data is: Collected Formatted Stored in a single, centralized location BAS MIS C.I. For customer reporting and analysis.
11 Data Warehouse Feed Components Store Use Data ETL Data Preparation Data Data Warehouse User Tools (Business Objects)
12 Business Objects Introduction Data Warehouse Power Suppliers Load BAS MIS Extract-Transform -Load Data ~1200 data elements Transactions Virtual Market Hierarchy Etc. Universe: View of a functional set of data within the warehouse Classes/Sub-classes: Groups of objects Objects: Individual data elements (for reporting and filtering) Example: Organization, Interval Start Time
DSS Project Summary Contains 99% of NYISO Settlement Billing Codes and all related Dimensional Data elements Power Suppliers Day Ahead & Balancing Market Energy and Bid Production Cost Guarantees Ancillary Services (Black Start, VSS, Regulation, Operating Reserves) Load Serving Entities Day Ahead & Balancing Market Energy Ancillary Services, Residuals, Supplier Uplift Allocations, and Other Charges Transaction Customers Day Ahead & Balancing Market Energy Day Ahead & Balancing Market Replacement Energy Day Ahead & Balancing Market Transmission Usage Charges Day Ahead & Real-Time Bid Production Cost Guarantee Ancillary Services, Residuals, Supplier Uplift Allocations, and Other Charges Virtual Market Day Ahead & Real Time Market Virtual Energy Transmission Congestion Contract Customers Rent Day Ahead Demand Response Program Customers Transmission Owners DAM Congestion Rent and NYPA NTAC 13
DSS Versioning Strategy BAS Settlement Volumes Advisory 1 DSS Settlement Versions Version 0 14
DSS Versioning Strategy BAS Settlement Volumes Advisory 1 Advisory 2 Advisory 3 Invoice (Advisory 3) Advisory 4 Advisory 5 Invoice (Advisory 5) Advisory 6 DSS Settlement Versions Version 0 Version 0 Version 0 Version 1 Invoice Version 0 Version 0 Version 2 Invoice??? 15
16 DSS Versioning Strategy BAS Settlement Volumes Advisory 1 Advisory 2 Advisory 3 Invoice (Advisory 3) Advisory 4 Advisory 5 Invoice (Advisory 5) Advisory 6 DSS Settlement Versions Version 0 Version 0 Version 0 Version 1 Invoice Version 0 Version 0 Version 2 Invoice Version 0 Key Point: Version 0, when it exist, represents the latest version of settlement data. If Version 0 does not exist, then the latest settlement will be the invoice with the highest Version Invoice #.
DSS Release Dates Availability dependent upon release dates. Balancing Market Settlements; July, 2003 DAM, Virtual and TCC markets; October, 2003 Power Supplier and LSE Ancillaries, Transactions and DADRP; December, 2003 As settlements prior to release dates are re-spun, additional data becomes available. Historic data will not be available 17
18 DSS Data Availability 2003 Present May June July Aug Sept Oct Nov Dec. Balancing Market Settlements Refer to: DSS Release Matrix Lists of Interval Days and Invoice Versions in DSS (Corporate Report Administrative) DAM, Virtual and TCC Markets Ancillaries, Transmission Owner and DADRP
DSS Corporate Reports Settlement Summary Reports Settlement Results Report Settlement Version Comparison Report Administrative Reports Settlement Version Control Log List of Interval Days & Invoice Versions Contained in DSS Settlement Details Reports One per market settlement for all NYISO Markets 19
20
21
22
23 New Report Wizard New Report New Report Manager - Visual Refresh Data Rerun/Prompts
Business Scenarios Instructor Led and Hands On Exercises
Business Scenarios Corporate Reports Scenario (#1A) - Review Settlement Results Scenario (#1B) - Compare Invoice Versions Scenario (#1C) - Review Power Supplier DAM Energy Settlement Details
Conceptual Data Models
27 Conceptual Data Models Document the primary relationships between related data entities Can assist with identifying the causes of Cartesian Product error messages
28 Power Supplier Conceptual Model Zones DAYS Subzones Billing Versions HOURS Generators GENERATOR FACTS SCD INTERVALS TIME Organizations
29 LSE Conceptual Model Zones SubZones Billing Versions DAYS Load Buses LOAD FACTS HOURS Load Serving Entities Organizations SCD INTERVALS TIME
30 Transaction Customers Conceptual Model ZONES SUBZONES ZONES Billing Versions DAYS LOAD BUSES SUBZONES TRANSACTION CONTRACTS TRANSACTION CUSTOMER FACTS HOURS LOAD SERVING ENTITIES GENERATORS SCD INTERVALS TIME ORGANIZATIONS ORGANIZATIONS
31 TCC Customers Conceptual Model Billing Versions DAYS TCC TCC FACTS HOURS Organizations TIME
32 Virtual Market Customers Conceptual Model ORGANIZATIONS VIRTUAL BID ENTITY ZONE Billing Versions DAYS HOURS VIRTUAL SUPPLY BUS VIRTUAL LOAD BUS VIRTUAL MARKET FACTS SCD INTERVALS TIME
33 DADRP Conceptual Model LSE Organizations Load Serving Entities Billing Versions DAYS Demand Response Buses DADRP FACTS HOURS TIME DR Provider Organizations
34 Transmission Owners Conceptual Model Billing Versions DAYS Transmission Providers TO FACTS HOURS Organizations TIME
Business Objects Universe Overview
Business Objects Universe Overview What is a Universe? The approach used to present the custom querying and reporting environment to users in a simplified manner. Logical grouping of data elements to make custom querying quicker and easier Universes High Level Organization Classes / Sub-Classes More focused but still high level Objects detailed level of data (actual data elements) 36
37 BO Universe High Level Design Within the many DSS BO Universes, data elements are initially broken down into 2 types: Dimensions Data elements that describe the data Common Dimensions (across Universe)» Date/Time» Invoice Version» Etc. Universe Specific» Generators» Load Serving Entities» Transaction Contracts» TCCs Facts Data elements containing desired values (actual values) For example: Scheduled Energy, Market Prices, Basepoints, Load Forecasts, etc.
38 Dimensions Facts by settlement
39 BO Universe High Level Design Factual data elements are organized in a manner to highlight the settlement algorithm Initially they are identified by settlement DAM Energy for Power Suppliers Balancing Market LBMP Energy for Transaction Customers TCC Rent Etc. Next they are broken down by time granularity (daily, hourly, SCD level) Then they are broken down into 4 categories based on purpose in the settlement Settlement Results Intermediate Calculations Billing Determinants Other Related Data Elements
40 Settlement Algorithm Stmt$ = MW x Price MW, Price Stmt$ Billing Determinants Settlement Results
41 Settlement Algorithm Stmt$ = MW x Price MW, E, L, C Billing Determinants Intermediate Calculations Price = E+L-C Stmt$ Settlement Results DAM Gen Bid Other Data Elements
42 Dimensions Facts by time granularity Facts by settlement
43 Dimensions Facts by time granularity Facts by settlement
44 Dimensions Facts
Business Objects Demo
Custom Query Creation 46
Custom Query Creation 47
Custom Query Creation Main Query Panel 48
Custom Query Creation Defining the Result Set 49
Custom Query Creation Defining the Query Conditions (data set) 50
Custom Query Creation Initial Report Results 51
Custom Query Creation Custom Report Formatting 52
53 - This is where you choose what data to bring in and build your report Show/Hide All Classes Show/Hide Help on Selected Item Wrap Results Object
54 Refresh Data Rerun/Prompts Edit Data Provider Query Panel View Data Purge prior to Save Slice and Dice Format Data
55 Show/Hide Available Variables Show/Hide Section Section Break Show/Hide Status Bar Apply Break Apply Sort Insert Calculation - This is where you change the format of your report, including creating sections and calculations
Training Environment Data Data has been masked to maintain confidentiality! Data available: September 2003 Invoice Versions 1 & 2 (1 is a partial set) 56
Business Scenarios Instructor Led and Hands On Exercises
58 Business Scenarios Basic Custom Queries & Reports Scenario (#2A) - Power Supplier Asset Management Bid Strategy Analysis Scenario (#2B) - Load Serving Entity Energy Procurement Strategy Analysis Scenario (#2C) - Transaction Customer Management Reporting Scenario (#2D) - Load Serving Entity - Ancillary Service Analysis Scenario (#2E) Power Supplier Ancillary Service Summary
59 Gotchas Here are a few scenarios to watch out for! In general: Failure to define appropriate conditions Failure to include the appropriate dimensional data elements in your result set Including mixed time-granularities in your result set Building queries with incompatible objects (Cartesian products) Slow performing queries due to inefficient query design
60 Saving, Printing, Exporting Once a report has been created, it can be: Saved as a Business Objects Report (template) Saved As a report with data and formatting Printed to hardcopy Exported to other applications (i.e. Excel)
61
62 BO Automatic Summation Feature Pink Circles are the summable objects that will be automatically rolled-up Blue Boxes are the data elements that define the lowest level dimensional grain Note: Summable (pink circles) versus Non-summable (blue boxes) is not exactly the same as factual and dimensional.
63 BO Automatic Summation Feature ( pink circles and blue boxes ) Business Objects will automatically roll-up data in your query to the lowest level of dimensional data granularity
64 BO Automatic Summation Feature SCD If your data is from the SCD-level The lowest dimensional grain in your query is hourly SCD data will be summed and Hour reported at the hour.
65 BO Automatic Echo Effect Day If your data is from the Day-level The lowest dimensional grain in your query is hourly Hour Day data will be echoed and reported the day value for each hour.
66 Understanding Granularity Organizational / Geographical Hierarchy Time Hierarchy Sum. Data
Understanding Granularity 67
68
Business Scenarios Instructor Led and Hands On Exercises
70 Business Scenarios Intermediate Custom Queries & Reports Scenario (#3A) - Summarize LSE Balancing Market Settlement Results Scenario (#3B) - Summarize Transaction Contracts Scenario (#3C) - Summarize PS Balancing Market Settlement Results
71 Avoid Gotchas Define Appropriate Conditions Include Appropriate Dimensional Data Elements In Your Result Set Avoid Mixing Time-granularities In Your Result Set Build Queries With Compatible Objects (Avoid Cartesian Products Markets vs Zones) Query Using Proper Granularity Check DSS Release Dates, Versions Practice - Practice
72 Additional Intermediate Features (Time Permitting) Multiple tables and tabs in a single report. Data can be presented graphically (charts and graphs). Reports can be created with data from multiple universes. Data can be calculated from the data elements provided.