CMS Approval: Philips CityTouch with Starsense PLC

Transcription

1 101/02 CMS Approval: Philips CityTouch with Starsense PLC Meeting Name Unmetered Supplies User Group (UMSUG) Meeting Date 12 April 2011 Purpose of paper Summary For Decision ELEXON has witness-tested the CityTouch with Philips Starsense (Power Line Control PLC) Central Management System (CMS) on the 24 April The CMS passed the witness testing. This paper presents the Test report and invites the UMSUG to recommend to the SVG that the system be approved for use in Settlement. 1. What are we seeking from the UMSUG? 1.1 The UMSUG is invited to review test report provided by Philips and the report on the witness testing from ELEXON. 2. What is CityTouch with Starsense PLC 2.1 Philips CityTouch is a sophisticated cloud-based CMS for tele-managing street lights in urban areas. Philips CityTouch can be configured to work with the Philips Starsense PLC and Starsense RF control systems. However, for this testing undertaken is approval for its use with the Philips Starsense PLC control system. This system is based on the LONWORKS protocol over powerline and uses mains borne signalling to communicate with tele-managed nodes. 2.2 We are seeking approval of the Power Line Control (PLC) version of the system (Version 1.3). Further incremental testing on the RF functionality will be undertaken on the CityTouch with the Starsense (RF) will be undertaken shortly and a new version (Version 1.4) will be presented for approval. 2.3 The system architecture is defined in the attached test report from Philips. It will interface with Customer s own inventory databases (e.g. Mayrise or Confirm). It is intended that the interface will be electronic at some point in the future. 3. Witness Testing 3.1 ELEXON has witness-tested the system and can confirm that all the testing was successful completed. Furthermore, Power Data Associates have confirmed that the event logs can be accessed and are in the correct format for use with Lailoken (CMS version). UMSUG101/02 v1.0 Page 1 of 2 ELEXON 2011

2 101/02 4. Recommendations 4.1 We invite you to: a) REVIEW Philips City Touch test report; b) NOTE that the system has passed the ELEXON witness testing; c) NOTE the further testing for the RF version of the system; and d) RECOMMEND the Philips City Touch with Starsense PLC (Version 1.3) be approved by the SVG for use in Settlement. Attachment: Attachment A Philips City Touch test report For more information, please contact Kevin Spencer Market Analyst [email protected] UMSUG101/02 v1.0 Page 2 of 2 ELEXON 2011

3 UMSUG_101/02 - Attachment_A Test Evidence Report for Philips CityTouch with Starsense PLC Date: 25/03/10 Version Number: 1.1

4 Contents Introduction...1 Test Hardware...2 Test Procedures...5 Appendix A...10 CityTouch Test Group Revision Control...11 Development Methodology and Quality Control...12 CityTouch Test Group CityTouch Test Group CityTouch Test Group 4 and Inventory Audit Trail...29 CityTouch Test Groups 6 and Dimming Shapes...31 Lamp Configuration and Dimming Shape Assignment...34 Scenarios 3 and Scenarios 1 and Switching-point Audit Trail...39 CityTouch Test Group Event Log Audit Trail...44 CityTouch Test Group Time taken to generate a daily event log for a large data set...46 Time taken to retrieve 10 months worth of switching point data for a light point...46 Time taken to add 3000 switching points...46 Time taken to add 648,450 switching points...47 CityTouch Test Group Independent event log file validation with Power Data Associates...49 Appendix B...50 References...52

5 Introduction This document is a test evidence report for the Philips CityTouch Central Management System (CMS), which is seeking approval for its use in Great Britain as the CMS component of an Equivalent Meter (EM). Philips CityTouch is a sophisticated cloud-based CMS for telemanaging street lights in cities. With Philips CityTouch, you can finesse the control of groups of street lights, or even of an individual street light. Philips CityTouch can be configured to work with the Philips Starsense PLC and Starsense RF control systems. However, for this test, we are seeking approval for its use with the Philips Starsense PLC control system. This system is based on the LONWORKS protocol over powerline and uses mains borne signalling to communicate with telemanaged nodes. Figure 1 outlines the architecture of Philips CityTouch. As you can see, it is has an independent core system, which defines an interface that control systems must implement to facilitate communication and interoperability. The Philips Starsense PLC and Starsense RF control systems implement this interface, and can therefore be plugged into Philips CityTouch to form a complete functional system. Driver independent implementation of Driver Interface in Core System Driver dependent implementation of Driver Interface in System Driver Task Framework, library is shared by all Drivers Philips Starsense PLC System Driver TF Philips CityTouch Core System Scope of Starsense PLC Control System Philips Starsense RF System Driver TF Network Nodes Ballasts & Lamps Network Nodes Ballasts & Lamps Figure 1: Architecture of Philips CityTouch For the sake of brevity, the term CityTouch is used in this document, to refer to the complete Philips CityTouch with Starsense PLC system. 1

6 Test Hardware The table below lists the hardware devices used for the tests. Device Name Lamp Type Charge Code Philips BGS451 LED 24W 1-10V /PH/EMEA/BGS451LEDStree424W/24/LEDSTREET Philips Dali Cosmo White 60W /PH/EMEA/COSMOWHITECPOTW/60/HIDPVXTSCPOTW Philips Dali Cosmo White 140W /PH/EMEA/COSMOWHITECPOTW/140/HIDPVXTSCPOTW Netmodul Netbox 2500 Router Not applicable Application pending Philips LFC 7065 Segment Controller Not applicable Application pending Philips LLC 7040 OLC Not applicable Philips LLC 7030 OLC Not applicable Philips LLC 7030 OLC Not applicable More details are provided in the CityTouch Test Groups 6 and 7, regarding the configuration of the lamps and control devices, with respect to dimming schedules, Switch Regime, CMS Unit Reference, and the other properties mentioned in Test Groups 4b and 5 of the Test Specification 2. With regard to the lamps, CityTouch uses dimming curves provided by the manufacturer (Philips), to compute the percentage of base power for switching points (in the event logs), as a function of the recorded dimming value. As an example, Figures 2 and 3 show sample dimming curves for a High Intensity Discharge (HID) lamp and Light Emitting Diode (LED) or fluorescent lamp respectively. They show the typical response characteristics of HID and LED or fluorescent lamps; for example, that LED and fluorescent lamps can be dimmed to a much lower level than HID lamps. 2

7 Figure 2: Typical dimming curve for an HID lamp 3

8 Figure 3: Typical dimming curve for an LED or fluorescent lamp 4

9 Test Procedures The tables below summarise all the test groups which are evidenced in this report. The test requirements are taken from the Test Specification 2. Test Group Test Requirements Test Evidence Test Group 1 System Configuration The operator of the CMS System should demonstrate the software versioning and operating platforms that will be subject to approval. Please see Appendix A - CityTouch Test Group 1 Test Group Test Requirements Test Evidence Test Group 2 Security The operator of the CMS System should demonstrate the procedures which provide secure access to data. Operators should only be able to access data which is relevant to them. Secure access procedures should be demonstrated for the following participants: Please see Appendix A - CityTouch Test Group 2 HHDC Suppliers Customers Test Group Test Requirements Test Evidence Test Group 3 Synchronisation to UTC The operator of the CMS System should demonstrate that the CMS System is Synchronised to UTC, either by connection to internet time servers or a radio clock, and are accurate to within ± 20 seconds per month. Please see Appendix A - CityTouch Test Group 3 5

10 Test Group Test Requirements Test Evidence Test Group 4 Inventory control information The operator of the CMS System should demonstrate the addition, modification and deletion of Inventory Control information required for the key test scenarios, either manually or electronically. The Data subject to testing is: Please see Appendix A - CityTouch Test Groups 4 and 5 Sub-Meter ID Effective From Data CMS Unit Reference Number of Items Switch Regime Charge Code The operator of the CMS System should demonstrate the recording of the audit trail for the entries made above. Test Group Test Requirements Test Evidence Test Group 5 Equipment control information If applicable the operator of the CMS System should demonstrate the addition, modification and deletion of Equipment Control information required for the Scenarios described in Section 4, either manually or electronically. The Data subject to testing is: Please see Appendix A - CityTouch Test Groups 4 and 5 CMS Unit Reference Sum of CMS Controller devices Switch Regime Charge Code The operator of the CMS System should demonstrate the recording of the audit trail for the entries made above. 6

11 Test Group Test Requirements Test Evidence Test Group 6 CMS Issue Instructions The operator of the CMS System should demonstrate the issuing of operational switching times and power level instructions for CMS Units in the CMS System for the following scenarios: Scenario 1 Switch Regime 999; Scenario 2 Switch Regime 998; Scenario 3 Control Failure (no data for a CMS Unit); Scenario 4 - Revised Data after control failure (following day) Please see Appendix A - CityTouch Test Groups 6 and 7 Test Group Test Requirements Test Evidence Test Group 7 Record operational switching times and power levels The operator of the CMS System should demonstrate the recording of operational switching times and power levels for CMS Units in the CMS System for the following scenarios: Scenario 1 Switch Regime 999; Scenario 2 Switch Regime 998; Scenario 3 Control Failure (no data for a CMS Unit); Scenario 4 - Revised Data after control failure (following day) The operator of the CMS System should demonstrate the audit trail for the above scenarios. Please see Appendix A - CityTouch Test Groups 6 and 7 7

12 Test Group Test Requirements Test Evidence Test Group 8 Generate Operational Event Log - normal processing and control failure The operator of the CMS System should demonstrate the sending of daily operational event logs of the operational switching times and power levels for specified CMS Units to the MA in the specified format for the following scenarios: Scenario 1 Switch Regime 999; Scenario 2 Switch Regime 998; Scenario 3 Control Failure (no data for a CMS Unit); Scenario 4 - Revised Data after control failure (following day) The operator of the CMS system should demonstrate a control failure (no data for a CMS Unit) through use of the correct information flag as per Scenario 3. Operational Event Log revision to previously reported data The operator of the CMS System should demonstrate that data can be revised by either issuing a refresh or an incremental operational event log (CMS and MA Separate Systems) to the MA in the specified format or if applicable the transferring of revised data (CMS and MA integrated System) from a previous control failure. (Scenario 3) The Refresh or Incremental Flow should cover: Please see Appendix A - CityTouch Test Group 8 Refresh Flow o A complete refresh of the operational event logs which includes previously unknown data o A complete refresh of the operational event logs which includes data which has been amended. Incremental Flow o An incremental update of the operational event log which includes previously unknown data o An incremental update of the operational event log which includes data which has been amended. The operator should demonstrate that the operational event log has been retained for audit purposes and that the audit trail is correct. 8

13 Test Group Test Requirements Test Evidence Test Group 9 Volume and Performance The operator of the CMS System should provide evidence of volume and performance tests completed by the applicant as part of their system testing, to the accredited test agent so that compliance with operational timescales can be assessed. Please see Appendix A - CityTouch Test Group 9 Test Group Test Requirements Test Evidence Test Group 10 Operational Event Log File format The operator of the CMS System should demonstrate that the operational event logs are in the specified format, as per BSCP520 1 Section (c). Please see Appendix A - CityTouch Test Group 10 9

14 Appendix A The sections below contain the detailed test evidence in this report. CityTouch Test Group 1 CityTouch_TestGroup1_020211_1_Test1.2 (CMS Operating Platform and Version) CityTouch is a sophisticated cloud-based software service, which has both client and server-side components. On the server-side, CityTouch has a heterogeneous server farm, comprised of 64-bit servers running Windows Server 2008 R2 SP1 and Ubuntu Linux 10.04, in the Amazon Elastic Compute Cloud (EC2). One of the Windows machines is a Primary Domain Controller (PDC). Other Windows machines act as database servers (running PostgreSQL 8.3) and as application servers (running NHibernate and Microsoft.NET Framework 4). The client-side component is a Silverlight 4 application. The client machine must have the Silverlight 4 Runtime installed and in addition must meet the following minimum requirements: Operating System Windows 7, Windows Vista, or Windows XP Service Pack 2 CPU Intel Core Duo 2.2 GHz or faster RAM For Windows XP Service Pack 2: 2GB For Windows 7 or Windows Vista: 4GB or more Internet Connection 2MBit/s or faster Browser Windows Internet Explorer 7, Windows Internet Explorer 8, or Mozilla Firefox 3.6 CityTouch_TestGroup1_020211_1_Test1.1 (CMS Software Version) The CityTouch software version is displayed on the application s About page. You can access this page by clicking the About link on the CityTouch log-in page (see Figure 9 in section CityTouch Test Group 2). Figure 4 shows a detail from the CityTouch About page. 10

15 Figure 4: Detail from the About page As you can see, the version number ( ) is represented by three numbers, separated by dots (.). The first is the major release number. The second is the minor release number and the third is the current Subversion (our revision control system) revision number. A new major release denotes a substantial overhaul of the software. When a new major release occurs, Philips will contact Elexon to arrange for the re-certification of CityTouch. A minor release denotes the addition of a new feature, such as a Works Order Management sub-system. A new Subversion revision will appear in the version number every three weeks and denotes a small increment to the product (such as a set of bug-fixes). Revision Control We use Subversion 1.5 ( as our revision control system. We have Subversion servers in both the Amazon cloud and our local development environment, which are kept in synch. Figure 5 shows the TortoiseSVN tool ( being used to browse our CityTouch Subversion repository. We also use the AnkhSVN ( plug-in, to access the repository within Microsoft Visual Studio In addition to simple revision control, we also use Subversion to enact sophisticated branching and merging strategies. For example, we may develop a new feature or component on a branch, before merging it back into the trunk prior to release. 11

16 Figure 5: TortoiseSVN browsing the CityTouch Subversion repository Development Methodology and Quality Control The CityTouch development team uses a range of cutting-edge development methodologies and test strategies, to ensure the quality of the released service remains high. The development team uses the Scrum agile development methodology. All code is developed (test-first) using the Test-Driven Development (TDD) methodology. This ensures code is well factored and of high quality, and facilitates (as a bi-product) the creation of a large suite of regression tests which can be run at any time. Currently, CityTouch has 328 test fixtures, comprising 2,520 separate unit tests, testing all aspects of CityTouch. Developers regularly run the fixtures relating to the areas they are working on. The CityTouch team also uses the Continuous Integration (CI) methodology with CruiseControl.NET ( to ensure CityTouch is built automatically (on a local build server) at least once daily and that the entire suite of unit tests are also run at least once daily. CI enables us to identify bugs and regression issues soon after they occur, so they can be fixed immediately 12

17 and don t become an established part of the code-base. Figure 6 shows the test results for an automated build on CruiseControl.NET. As you can see, for the 2,520 tests there were 0 test failures. There were 20 ignored tests (indicated in yellow in the dashboard display), but these can run only on developer machines (not on the build server). Figure 6: Test results for an automated build in CruiseControl.NET In addition to the 2,520 automated tests mentioned above, a battery of manual integration tests and smoke tests are run (and must be passed) prior to any production release. These tests are defined in written test scripts. 13

18 CityTouch Test Group 2 CityTouch_TestGroup2_180211_1 (System Security) Test References: 2.1, 2.2, 2.3, and 2.4 CityTouch has a comprehensive role-based security implementation, to enables users to authenticate themselves and authorise access to various parts of the application. In CityTouch, users with the role Master can access the administration site. On the Users page, they can add other users to CityTouch. Figure 7 shows a detail from the Users page. Figure 7: Detail from the Users page When the Create User button is clicked, a Create User page appears. Master users can then add the details of the user and specify their role(s). The user name must be a valid address. Figure 8 shows a detail from the Create User page. Figure 8: Detail from the Create User page 14

19 As you can see, there are many roles in CityTouch. Users can have several roles, and each role enables access to different parts of the application (with read or write access). For example, the User role has read-only access to the application, but the Operator role can create dimming calendars and apply them to light points. New read-only users (with just the User role) can be created as required. The read-only user indicated in Figure 8 was created and Figure 9 shows this user logging-in (using the CityTouch log-in page). Figure 9: Read-only user logging-in Figure 10 shows the read-only user logged into CityTouch. There is also an UnmeteredSupplies role, which lets users view the Unmetered Supplies page on the CityTouch administration site. This page lets users generate and send event logs, and view the various types of audit trails. Figure 11 shows an UnmeteredSupplies user logged into the Unmetered Supplies page. 15

20 Figure 10: Read-only user logged-in to CityTouch 16

21 Figure 11: UnmeteredSupplies user on the Unmetered Supplies administration page 17

22 CityTouch Test Group 3 CityTouch_TestGroup3_150211_1 (Synchronisation to UTC) CityTouch servers and control devices are synchronised with NTP time servers, to ensure they always use UTC time accurate to within a few milliseconds. The Primary Domain Controller (PDC) is synchronised with an NTP time server, and the PDC synchronises, in turn, the time of its associated database and application servers. Figure 12 is a Command Prompt, which shows that the PDC is synchronised with 1.europe.pool.ntp.org, and that its current time drift is only 9 milliseconds. Figure 12: Time synchronization and drift of PDC CityTouch Linux servers are also synchronised with NTP time servers. Figure 13 shows the ntp.conf file of one of the Linux servers. As you can see, the file specifies four time servers. The Linux server will try to synchronise with 0.europe.pool.ntp.org first, but if that s unavailable, it will endeavour to synchronise with the next server in the list (and so on). 18

23 Figure 13: Linux server NTP configuration Philips Starsense PLC devices are also synchronised to NTP time servers. Figure 14 shows a configuration page of a Segment Controller, which shows its assigned time server. It synchronises with the time server over GPRS. The Segment Controller synchronises time with its associated Outdoor Lamp Controllers (OLCs). This is particularly important as the Segment Controller and OLCs log switching points. They log switching points in local time (e.g. GMT or BST), but always record the offset from UTC, so we always compute correct UTC times for the event logs. The IP address of the time server is set initially when the system is commissioned. 19

24 Figure 14: Segment controller time synchronisation configuration 20

25 CityTouch Test Group 4 and 5 CityTouch_TestGroup4b_180211_1 (Inventory Control Information) Test References: 4.3, 4.3.1, 4.3.2, 4.3.3, 4.3.4, 4.3.5, and CityTouch_TestGroup5_180211_1 (Equipment Control Information) Test References: 5.1, 5.1.1, 5.1.2, 5.1.3, and The current version of CityTouch has limited asset management capabilities. However, it does manage the properties mentioned in Test Groups 4b and 5 of the Test Specification 2, as well as some other general asset properties. It does not manage all the properties mentioned in Test Group 4a, nor does it contain functionality for exporting a Detailed Inventory or Control File. In terms of unmetered supplies data flows, CityTouch fulfils only the Central Management System (CMS) role, shown in Figure 15. Figure 15: Data flows between unmetered supplies roles For the purposes of this test, we do not wish to characterise CityTouch as an Asset Management System (AMS), but rather as a fully-featured Telemanagement System. In CityTouch, you can add lights and control devices with a Commissioning Action. A Commissioning Action is a link on the User Interface (UI), which enables you to upload an XML file with the associated asset and network topology data. Figure 16 shows CityTouch with an empty Light Points asset grid (prior 21

26 to commissioning). The Network Nodes grid (for control devices) is also empty (see Figure 17). The user must have the Configurator role, to view (or click) the Commissioning Action and the reciprocal Delete Action (explained later). Figure 16: Empty light point s grid prior to commissioning 22

27 Figure 17: Empty network nodes grid prior to commissioning To add lights and control devices, you select the PhilipsStarsensePLC provider in the Network Tree and click the Add Segment action link. Figure 18 shows the selected provider and action link. When you click on the link, CityTouch displays a File Upload dialog, to enable you to select and upload the XML file. Figure 19 shows the File Upload dialog. Figure 18: Selected provider with the Add Segment action link 23

28 Figure 19: File Upload dialog When you click on the Next button in the File Upload dialog, the asset and network topology data is loaded into CityTouch and the system is commissioned. Figure 20 shows CityTouch with a populated Light Points grid (after commissioning). The Network Nodes grid is also populated (see Figure 21). 24

29 Figure 20: Populated light points grid after commissioning 25

30 Figure 21: Populated network nodes grid after commissioning At this point, none of the data for Sub-Meter ID, CMS Unit Reference, Effective From, Switch Regime, Charge Code, and Number of Items exists. However, columns for these items (mentioned in Test Groups 4b and 5) appear in both the Light Points and Network Nodes grids. To add or update data in these columns, you simply select a column cell and type in the value. Validation exists on all these columns, to ensure the data corresponds to the format specified in the Operational Information Document A Guide to Unmetered Supplies under the BSC 3. Figure 22 shows the Light Points grid. Data has been added for the topmost asset and the Sub-Meter ID for the second asset is currently in the process of being edited. The same test was performed for the Network Nodes grid (see Figure 23). Figure 22: Adding and updating the properties in Test Group 4b 26

31 Figure 23: Adding and updating the properties in Test Group 5 For assets to have corresponding entries in the event logs, they must have a valid Sub-Meter ID and CMS Unit Reference, and their Effective From date must before or on the event log date. To delete assets in CityTouch, you use a reciprocal action to the Commissioning Action a Delete Action. As with the Commissioning Action, a Delete Action is also a link on the UI. To delete assets, you select a segment controller associated with the PhilipsStarsensePLC provider in the Network Tree and click the Delete action link. Figure 24 shows the selected segment controller and action link, and a message box asking whether you really wish to perform the deletion. The action deletes all assets directly or indirectly associated with the segment controller (e.g. OLCs, lamps, and the segment controller itself). Figure 24: Selected segment controller with Delete action link Figure 25 shows CityTouch with an empty Light Points grid after the Delete Action was invoked. The Network Nodes grid is also empty (see Figure 26). 27

32 Figure 25: Empty light points grid after asset deletion 28

33 Inventory Audit Trail Figure 26: Empty network nodes grid after asset deletion CityTouch_TestGroup4b_180211_1_Test4.4 (Audit Trail) CityTouch_TestGroup5_180211_1_Test5.2 (Audit Trail) The actions described above, to add, update, and delete data about lights and control devices, generate audit trail items which can be viewed in CityTouch. To view this audit trail, go to the Unmetered Supplies page on the CityTouch administration site, where you will see the Detailed Inventory Audit Trail UI shown in Figure 27. Figure 27: Inventory audit trail UI 29

34 You can the select the Site the inventory is associated with, as well as From and To dates (either or both of these fields can be empty, to make the date range unbounded on one or both sides). When you click the Generate button, CityTouch will generate and download a spreadsheet of all audit trail items in the specified date range. Figure 28 shows the audit trail for the adding, editing, and deleting of inventory properties performed in this section. Figure 28: Inventory audit trail after adding, editing, and deleting As you can see, the audit trail shows the: CMS Unit Reference, The user name of the person who made the change The date of the modification The type of modification: INSERTED, UPDATED or DELETED The property that changed (if any) and the value it changed to A separate audit trail item is generated for each cell edit in the asset grids. 30

35 CityTouch Test Groups 6 and 7 CityTouch_TestGroup6_080311_1 (CMS Issue Instructions) Test References: 6.1, 6.2, 6.3, and 6.4 CityTouch_TestGroup7_080311_1 (Record operational switching times and power levels) Test References: 7.1, 7.2, 7.3, and 7.4 Dimming Shapes In CityTouch, you can define dimming shapes and apply them, using rules and calendars, to light points. For Test Groups 6 and 7, we defined two dimming shapes, corresponding to the patterns defined in Scenario 1 Switch Regime 999 and Scenario 2 Switch Regime 998, in sections 4.1 and 4.2 respectively in the Test Specification 2. Figure 29 shows the dimming shape for Scenario 1 Switch Regime 999 and Figure 30 shows the dimming shape for Scenario 2 Switch Regime 998. Figure 29: Dimming shape defined for Scenario 1 Switch Regime

36 Figure 30: Dimming shape defined for Scenario 2 Switch Regime 998 Figure 31 shows a detail from the Switching Points grid for Dimming Shape 999. The switching points match the ones specified in section 4.1 of the Test Specification 2. Figure 31: Switching points for Dimming Shape 999 Figure 32 shows details of the Switching Points grid (with scrollbar up and down) for Dimming Shape 998. The switching points have a similar shape to the ones specified in section 4.2 of the Test Specification 2, 32

37 except that we scaled them to have a 50% minimum (rather than 0%), because we feel a 0% value contradicts the notion of an always-on regime. Figure 32: Switching points for Dimming Shape 998 We also defined a second always-on (998) dimming shape (with just one switching point), to demonstrate that, when applied to a lamp, only one switching point is added to the daily event log (in the non-failure scenario); it shows that the lamp is recorded as on, even though the initial instruction to turn on may be have been issued sometime in the past. Figure 33 shows this always-on dimming shape. Figure 33: Second always-on (998) dimming shape 33

38 Lamp Configuration and Dimming Shape Assignment Figure 34 shows the lamp configuration for the tests. All the properties mentioned in Test Group 4b of the Test Specification 2 have values. Please note that the Cosmo White 60W and BGS451 LED 24W 1-10V have Switch Regime 998, while the Cosmo White 140W has Switch Regime 999. All three lamps have separate Sub-Meter IDs to fulfil the criterion mentioned in Section 4.3 of the Test Specification 2, that we can reproduce failures across multiple unmetered supplies. It also ensures that separate event logs are generated for each lamp (to clarify comprehension of the test results). Figure 34: Lamp configuration for the tests Figure 35 shows which dimming shapes are applied to which lamps (via the associated colours). Dimming Shape 999 is applied to the Cosmo White 140W, Dimming Shape 998 is applied to the Cosmo White 60W, and the second always-on dimming shape is applied to the LED lamp (Ledgine 24W). Figure 35: Lamp configuration for the tests For the tests, all three lamps were set to burn continually using the assigned dimming shapes for several nights. Of course, during this time, the Cosmo White 60W and Ledgine 24W also burnt continually during the day. 34

39 Scenarios 3 and 4 This section documents the test evidence for Scenario 3 Control Failure for Multiple CMS Unit References and Scenario 4 Revised Data after Control Failure (Following Day). The tests were performed with the Cosmo White 60W and CosmoWhite140W lamps, which have 998 and 999 switch regimes respectively. We performed these tests over five consecutive days. The lamps had been burning (with their assigned dimming shapes) for several days prior to the tests. If a connectivity failure occurs in CityTouch, no switching points for the affected lamps will be added to the event logs until the correct switching points appear. If the correct switching points appear a day or more late, they will be added to refresh event logs. CityTouch does not amend switching points which were added to previous event logs. New versions of event logs always show the latest record of what happened for lamps associated with those Sub-Meters on those days. On Day 1 (4 th March 2011) of the test, we disconnected the data cable to the Segment Controller, to create a connectivity failure. The lamps, of course, controlled by their OLCs, continued to dim according to their assigned shapes. However, there was no communication at all between the Segment Controller and CityTouch, and no way to retrieve switching points. We also left the cable disconnected on Day 2 (to demonstrate a control failure over a range of settlement days ). On Day 3, we reconnected the cable. In terms of event logs, early in the morning of Day 2, CityTouch generated and sent empty event logs (apart from the file header and trailer) for the two lamps (for Day 1). It did the same early on Day 3 (prior to the reconnection), for Day 2. On Day 4, after the reconnection, it sent six event logs: two refresh event logs for Day 1, two refresh event logs for day 2, and two event logs for day 3. The refresh logs contain all the correct switching points for the respective days. Figures show these various event logs. Figure 36: Empty event logs sent on Day 2 (for Day 1) 35

40 Figure 37: Empty event logs sent on Day 3 (for Day 2) Figure 38: Refresh event logs sent on Day 4 (for Day 1) 36

41 Figure 39: Refresh event logs sent on Day 4 (for Day 2) 37

42 Scenarios 1 and 2 Figure 40: Event logs sent on Day 4 (for Day 3) This section documents the test evidence for Scenario 1 Switch Regime 999 and Scenario 2 Switch Regime 998. The tests were performed with the Cosmo White 60W and Cosmo White 140W lamps, with the configuration described in the Lamp Configuration and Dimming Shape Assignment section. The tests were performed on the day after the Scenario 3 and 4 tests. Early in the morning of Day 5, CityTouch generated and sent event logs for the two lamps (for Day 4). Figure 41 shows the event log for the Cosmo White 60W and Figure 42 shows the event log for the Cosmo White 140W. Figure 41: Event log for Cosmo White 60W with Dimming Shape 998 applied 38

43 Figure 42: Event log for Cosmo White 140W with Dimming Shape 999 applied As you can see, these event logs show correct switching points for Day 4. We performed another test for these scenarios using the LED lamp (Ledgine24W) with the second always-on dimming shape. This shape has just one switching point per day (at 75% dimming). Figure 43 shows the event log for this lamp s Sub-Meter, generated on Day 5 (for Day 4). It contains only one switching point, which corresponds to the switching point in the shape. Figure 43: Event log for Ledgine24W with second always-on (998) dimming shape Switching-point Audit Trail CityTouch_TestGroup7_090311_1 (Record operational switching times and power levels) Test References: 7.5 An audit trail item is generated in CityTouch, whenever a switching point is recorded. To view this audit trail, go to the Unmetered Supplies page on the CityTouch administration site, where you will see the Switching Points Audit Trail UI shown in Figure

44 Figure 44: Switching point audit trail UI To generate an audit trail, select the From and To dates, enter an Asset ID, and click the Generate button. CityTouch will download an Excel spreadsheet, containing all the associated audit trail items. The spreadsheet shows the: Asset ID Timestamp of the switching point (in UTC) Percentage of base power Percentage dimming Figure 45 shows the switching point audit trail for the Cosmo White 60W for the entire test run. 40

45 Figure 45: Audit Trail for the Cosmo White 60W (998) for Day 1 to Day 4 41

46 CityTouch Test Group 8 CityTouch_TestGroup8_090311_1 (Generate Operational Event Log) Test References: 8.1, 8.2, 8.3, 8.4, 8.5 As described in CityTouch Test Groups 6 and 7, CityTouch generates event logs each morning at a preconfigured time (currently set to 07:00am GMT [UTC]). It generates a separate event log per sub-meter per day, and sends these files immediately to folders on our FTP server (where they can then be picked up by the Meter Administrator s daily dial process). The folder names are Sub-Meter IDs. Figures 46 to 48 show the event logs, generated in CityTouch Test Groups 6 and 7, in their respective sub-meter folders on our test FTP server. Figure 46: Folder for sub-meter abcd456 42

47 Figure 47: Folder for sub-meter abcd789 Figure 48: Folder for sub-meter abcd123 43

48 As mentioned later in CityTouch Test Group 10, the British Meter Administrator (MA) Power Data Associates have verified in writing that they were able to successfully dial, retrieve, parse, and process the event logs generated in CityTouch Test Groups 6 and 7. In addition to the scheduled event log generation, UnmeteredSupplies users can generate and send adhoc event logs, using the Event Log UI on the Unmetered Supplies administration page (shown in Figure 49). Figure 49: Event log UI on the Unmetered Supplies administration page When the Upload to Meter Administrator button is clicked, CityTouch will generate and send new event logs for today to our FTP server (assuming new switching points have been received for the site since the last event log generation). It increments the version numbers correctly. The button click does nothing if no new switching points were received. In the left-hand part of the UI, if you select a Site / Sub-Meter ID combination and From date, and click the Generate button, CityTouch will generate and download an un-versioned event log for that day and sub-meter. The event log will show the latest and best understanding of what happened to the lamps associated with the sub-meter on that day. Clicking Generate does not send the event log to the FTP server. Event Log Audit Trail CityTouch_TestGroup8_090311_1 (Generate Operational Event Log) Test References: 8.7 An audit trail item is added to CityTouch, whenever an event log is generated and sent. To view this audit trail, go to the Unmetered Supplies page on the CityTouch administration site, where you will see the Event Log Audit Trail UI shown in Figure 50. Figure 50: Event log audit trail UI 44

49 To view an event log audit trail, select a Site / Sub-Meter ID combination and click the Generate button. CityTouch will download an Excel spreadsheet, containing all the associated audit trail items. The spreadsheet shows the: Shipping date Event log date Filename Version number User who generated and sent the event logs FTP status code - ClosingData indicates that the file was sent successfully Figure 51 shows the event log audit trail for sub-meter abcd789 during the tests in CityTouch Test Groups 6 and 7. Figure 51: Event log audit trail for sub-meter abcd789 45

50 CityTouch Test Group 9 CityTouch_TestGroup9_180211_1_Test9.1 (Compliance with operational timescales) CityTouch has received extensive volume and performance testing. The following metrics were devised to gauge the performance of CityTouch with large volumes of data in relation to event log generation, switching point audit data retrieval, and switching point insertion: Time taken to generate a daily event log for a large data set (64,845 light points 7 switching points per day) Time taken to retrieve 10 months worth of switching point data for a light point (which has 7 switching points per day) Time taken to add 3000 switching points Time taken to add 648,450 switching points Test machines The tests were performed on two test machines: Machine A: Machine B: 2.4GHz Intel Core 2 quad-core CPU, 4GB RAM, 200GB HDD, 64-bit Windows 7 Enterprise Intel Xeon E5405 CPU, 8GB RAM, 200GB HDD, 64-bit Windows Vista Time taken to generate a daily event log for a large data set This test was performed on a large data set, consisting of 64,845 light points (the number of light points in Washington DC). Each light point had 7 switching points per day and the test was performed on Machine A. It took 25 seconds to generate and download a daily event log. Time taken to retrieve 10 months worth of switching point data for a light point This test was also performed on the Washington DC data set (64,845 light points). Each light point had 7 switching points per day and the test was performed on Machine A. It took 3 seconds to retrieve 10 months worth of switching-point data for a given light point and to download the data in an Excel spreadsheet. Time taken to add 3000 switching points This test was performed on Machine B. It took 20 seconds to insert 3000 switching points, using the CityTouch core system Application Programmers Interface (API). 46

51 Time taken to add 648,450 switching points This test was performed on Machine B and entailed storing all the switching points for Washington in a single batch. In addition to storing 8 switching points for 64,845 light points it also stored 2 switching points for 64,845 associated OLCs, making 648,450 switching points in all. It took 70 minutes to store these switching points as a single batch. With regard to this test it should be noted that: CityTouch normally adds switching points in small chunks throughout the day, rather than as a single huge batch. CityTouch creates new PostgreSQL partial tables for switching points in new months automatically, to ensure database performance is not compromised by switching point tables becoming too big. The algorithm for adding switching points is necessarily complex, as de-duping must be done (to ensure duplicate points are not persisted) and error switching points must be updated if revised switching points are received. This requires additional table lookups per insertion. 47

52 CityTouch Test Group 10 CityTouch_TestGroup10_080311_1 (Operational Event Log) Test References: 10.1, 10.2, 10.3, 10.4, 10.5, 10.6, 10.7, 10.8, , and This section demonstrates that CityTouch generates event logs which meet exactly the criteria specified in BSCP520 1 Section (c). For the purposes of this test, we opened one of the event logs (abcd log) generated in CityTouch Test Group 7. Figure 52 shows this event log in a text editor. Figure 52: abcd log opened in EditPad As you can see, the first 7 characters in the filename are the Sub-Meter ID (abcd456), followed by the date in YYYYMMDD format ( ), the 3-digit version number (001), and the.log filename extension. The file header starts with the H identifier, followed by the 7-character Sub-Meter ID (abcd456), the date in YYYYMMDD format ( ), and the 3-digit version number (001). In the file body, each row starts with a 12-character CMS Unit Reference (CMSURDCW140W), followed by the UTC time in HHMMSS format (e.g ), and the percentage of base power in PPP.PP format (e.g ). The file trailer starts with the T identifier, followed by the total number of lines in the file (9 in this case) padded with leading-zeros in a 7-digit field (T ). 48

53 Independent event log file validation with Power Data Associates A CityTouch team member sent some control files, corresponding to the event logs generated in CityTouch Test Groups 6 and 7, to the British MA Power Data Associates. Power Data Associates were then able to configure their MA system to successfully dial, retrieve, parse, and process the event logs generated in CityTouch Test Groups 6 and 7. Power Data Associates have confirmed this in writing, and their letter will be submitted to Elexon as a separate item of evidence. 49

54 Appendix B This appendix documents test evidence from the witness test, held at the Philips Centre in Guildford on the 24 th March It shows switching point audit trails for the Cosmo White 60W and Ledgine 24W, showing their dimming to 75% and 25% of light level respectively (with the associated percentages of base power). These audit trails are shown in Figures 53 and 54. Figure 53: Dimming of Cosmo White 60W the to 75% light level 50

55 Figure 54: Dimming of the Ledgine24W to 25% light level 51

56 References This document (including Appendix A) references the following documents: 1 Elexon Ltd (2010, November). BSCP520 - Balancing and Settlement Code Procedure - Unmetered Supplies Registered in SMRS, Version Retrieved from: 2 Elexon Ltd (2008, August). Central Management System Equivalent Meter Test Specification, Version 2.0. Retrieved from: 3 Elexon Ltd (2010, July). Operational Information Document A Guide to Unmetered Supplies under the BSC, Version 8.0. Retrieved from: 52