How to design a large scale Informix based Smart Facility / Smart Building Management Solution. Alexander Koerner IBM Germany

Transcription

1 How to design a large scale Informix based Smart Facility / Smart Building Management Solution Alexander Koerner IBM Germany 1

2 Alexander Celebrating 25 Years of Informix InfoWorld, Nov 13th, 1989 ( 2

3 My Project An Informix TimeSeries, REST API, JSON & Raspberry Pi 2 based Weather Station... 3

4 Acknowledgements and Disclaimers Availability. References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. The workshops, sessions and materials have been prepared by IBM or the session speakers and reflect their own views. They areprovided for informational purposes only, and are neither intended to, nor shall have the effect of being, legal or other guidance or advice to any participant. While efforts were made to verify the completeness and accuracy of the information contained in this presentation, it is provided AS-IS without warrantyof any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this presentation or any other materials. Nothing contained in this presentation is intended to, nor shall have the effect of, creating any warranties or representations from IBMor its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. All customer examples described are presented as illustrations of how those customers have used IBM products and the results theymay have achieved. Actual environmental costs and performance characteristics may vary by customer. Nothing contained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific sales, revenue growth or other results. Copyright IBM Corporation All rights reserved. U.S. Government Users Restricted Rights Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. IBM, the IBM logo, ibm.comare trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol ( or TM), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at Copyright and trademark information at Other company, product, or service names may be trademarks or service marks of others. 4

5 Agenda Background A typical Customer Profile Common Reasons to Act Anticipated Benefits The Solution Concept The Implementation Concept Informix Specifics Summary 5

6 Background 1/3 Companies become more and more energy efficiency aware Energy is an important cost factor as part of the industrial production process The utilization of renewable energy resources in buildings requires better monitoring, routing and control Special buildings like e.g. hotels or hospitals need fine granular energy consumption monitoring and optimizations 6

7 Background 2/3 Service providers who provide typical last mile relationships to their customers are more and more waking up and envison smart facility management offerings E & U Companies Telco Companies Monitoring Equipment Manufacturers Climate Control Providers The Key Ingredients of a Smart Facility Management Solution are Sensor Data and Facility/Building Meta Data 7

8 Background 3/3 Examples for typical Facility Sensor Data are: Temperature sensors in rooms On/Off status data of lights Status of Air Handling Units (AHUs) Photovoltaic roof energy production data People presence in rooms Examples for typical Facility/Building Meta Data are: Details on the current space/room usage (e.g. size) Location of the monitoring devices Room utilization (e.g. storage room, office, hospital room) Space/room ownership(s) 8

9 Agenda Background A typical Customer Profile Common Reasons to Act Anticipated Benefits The Solution Concept The Implementation Concept Informix Specifics Summary 9

10 A typical customer profile One of the world wide leading providers of secure, protected and energy efficient buildings and infrastructures Describes itself as a technology partner, product & service provider plus system integrator More than 100k employees worldwide More than 4 billion US$ revenue in 2013 Headquartered in Europe 10

11 Agenda Background A typical Customer Profile Common Reasons to Act Anticipated Benefits The Solution Concept The Implementation Concept Informix Specifics Summary 11

12 Common Reasons to Act The customer is currently in a phase of change He plans the introduction of extended and new services around Smart Facility management as a Service for more than 100k buildings in the mid term The Smart Facility Mangement market is already experiencing a very strong competition in which the customer plans to differentiate itself An existing Smart Facility Management in-house solution is lacking functionality and scalability To enhance the existing solution would be too complex and hence too expensive 12

13 Agenda Background A typical Customer Profile Common Reasons to Act Anticipated Benefits The Solution Concept The Implementation Concept Informix Specifics Summary 13

14 Anticipated Benefits 1/2 To be able to provide an innovative Smart Facility Management as a Service offering to a large number of customers Centralized collection and normalization of different data sources in one Smart Facility Data Respository Offering of added value services (analytics, reporting, benchmarking) to provide ways to cost optimize building operations Utilization of analytic methods, rule based models and tools to gain insight into the quality, breadth and relationships of the raw data 14

15 Anticipated Benefits 2/2 A continuous data flow of sensor data and meta data into the Smart Facility Repository will ensure near real time recommendations, optimizations and reports to their customers Support for all major standards in the area of asset data management and sensor data protocols/formats will guarantee easy data imports The highly scalable solution architecture will support the continuous growth with a focus of the anticipated > 100k buildings in the mid term 15

16 Agenda Background A typical Customer Profile Common Reasons to Act Anticipated Benefits The Solution Concept The Implementation Concept Informix Specifics Summary 16

17 The Solution Concept Integration of existing asset data, time series based sensor data and data from additional sources (e.g. meteorological data) in one repository Execution of thourough analysis, complex rules and report generation based on that one respository Phase-wise introduction of the new solution, while running the existing solution in parallel The operation model will be a multi tenancy, managed service in the cloud A licensable version for on premises use should be possible 17

18 Agenda Background A typical Customer Profile Common Reasons to Act Anticipated Benefits The Solution Concept The Implementation Concept Informix Specifics Summary 18

19 The Implementation Concept Phase-wise implementation of the new solution based on the following building blocks: Tivoli Maximo (asset management) IBM Informix Advanced Enterprise Edition (sensor data management) Optional: Informix Warehouse Accelerator (for complex analysis, benchmarks) IBM Cognos BI in combination with DataClarity s DashInsight (3rd party dashboards) IBM ILOG s ODM for complex rule execution plus IBM SPSS, InfoSphere DataStage, IBM TM1, ISR (BP) Billing Solution, IBM FileNet/Case Mgmt, IBM Connections, IBM Sametime, IBM Emptoris 19

20 Architecture Blueprint Mobile User Customers & external user IBM Worklight Mobile Platform Common UI DashInsight from Data Clarity Smart Facility Mgmt UI Common Security Single Sign On IBM Security Access Manager IBM Security Idendity Manager Sensor-/Device Mgmt UI Sensor-/Device-Management Buildings/Asset Mgmt IBM Maximo Assets Transport andintegration Layer IBM InfoSphere Smart Facility Mgmt UI IBM Cognos Emptoris Procurement Common Rules IBM ODM Analytics SPSS / TM1 Performance DB IBM Informix Performance Data Bill Mgmt. ISR APW Document Mgmt IBM FileNet Common Doc Mgmt Platform IBM FileNet Doc Archive Common Colaboration IBM Connections IBM Sametime Sensors Meters Elevators Fire HVAC Lighting Security 20

21 Agenda Background A typical Customer Profile Common Reasons to Act Anticipated Benefits The Solution Concept The Implementation Concept Informix Specifics Summary 21

22 Informix Specifics The Informix team (Lab, Services & Tech Sales) participated in a PoC (February 2014 April 2014) with a focus on sensor data management, Informix/Tivoli Maximo integration, Cognos BI reporting on sensor data and ILOG ODM access to sensor data PoC results customer presentation early IBM Ehningen (Germany) Informix Version used during the PoC: FC3 22

23 Initial Data Load (PoC) Data load from CSV files used a custom conversion program High speed conversion Able to handle multiple date formats Automatically calculate delta values Apply correction such as F->C or Wh->kWh Output in compact TimeSeries Loader format Informix High speed TimeSeries Loader used to insert data Time to load 10,200 meters with 27 months historical data 1hr 43minutes Predicted time to load daily data (1 new + 27 old days) < 4 minutes Important disclaimer: The PoC HW setup for the data load was not optimal, hence one should see those numbers only as an performance indicator Space used - historical data for 27 months 800 million TimeSeries elements 5.6 million 4K pages = 22.4GB Suggested relational size would be 35GB(w/o any secondary indexes!) A foreign key relationship between the meters table and the ifx_assets table ensures that only defined assets can be loaded. 23

24 Staged Sensor Data Loading (PoC) Three part process 1)Load 28 day data set into staging TimeSeries 2) Check and report integrity of staged data 3) Update readings with stage data based on chosen rules CSV Data Formatter Readings Reporting TSLoader Stage Data Update Integrity Check History 24

25 Custom Time Series Function 1/3 For the PoC we developed onecustom Time Series C-function (utilizing the Informix Time Series C-API, 951 lines of code partially re-used from another project) to handle the staging, data validation and correction plus the value historization: CREATE FUNCTION UpdateWithHistory ( update_flag mi_inetegr, target TimeSeries, stage TimeSeries, history TimeSeries, start DATETIME YEAR TO DAY, end DATETIME YEAR TO DAY DEFAULT NULL, correction DECIMAL) RETURNING INTEGER CREATE FUNCTION UpdateWithHistory ( update_flag mi_inetegr, target TimeSeries, stage TimeSeries, history TimeSeries, start DATETIME YEAR TO DAY, end INTEGER, correction DECIMAL) RETURNING INTEGER; This function takes the data values from a stage TimeSeries and transfers them to the target version of the TimeSeries. If the history TimeSeries is not null then it keeps an historical record of the update change. The stage time series is populated via Informix time series high performance loader 25

26 Custom Time Series Function 2/3 The row types, using DECIMAL for values, are: target - ROW (tstamp, status, value, delta, raw) stage - ROW (tstamp, status, value) history - ROW (tstamp, updatestamp, status, old, new) The history log can be kept in two ways: The history element timestamp is the same as the stage value tstamp. The update_stamp is set to be CURRENT. This is the default. The tstamp of the history element is set to be CURRENT. The update_stamp is set to the stage value tstamp. The status value for the target time series indicates if the data is estimated or a correction. 0x00 = Regular data value 0x01 = Estimated value from interpolation 0x02 = Correction from meter The status value for the stage time series indicates the change. 0x00 = No change 0x01 = Value estimated using interpolation 0x02 = Correction from meter, overwriting old data 0x03 = New data, overwriting NULL elements 0x04 = Overwrite old data The status value for the history indicates what change was made to the data. 0x01 = Estimated value from interpolation 0x02 = Correction from meter, overwriting old data 26

27 Custom Time Series Function 3/3 The update_flag controls the type of update 0x0000 = CHECK_ONLY - Only check the data, no updates 0x0001 = UPDATE_TARGET - Update the target values 0x0002 = UPDATE_STAGE - Update the stage status 0x0004 = UPDATE_HIST - Keep history of changes to target 0x0008 = INSERT_NEW - Copy across new values 0x0010 = CORRECT_ERRORS - Correct the errors 0x0020 = INTERPOLATE_FIX - Interpolate bad values 0x0040 = OVERWRITE_FIX - Overwrite bad values with last good value 0x0080 = OFFSET_FIX - Fixup the data by applying an offset 0x1000 = HISTORY_TS_CURRENT - The history TimeSeries tstamp is CURRENT 0x2000 = STORE_NULL_UPDATE - Keep history of NULL updates 27

28 Informix / Maximo Integration (PoC) A simple web service interface on top of Informix using Informix 4GL has been created during the PoC (but not utilized within the PoC context) in order to provide access to the sensor data <soapenv:envelope xmlns:soapenv=" xmlns:ws=" <soapenv:header/> <soapenv:body> <ws:ifx_sensor_request> <ws:building_id> electricityconsumption</ws:building_id> <ws:from> :00:00</ws:from> <ws:to> :45:00</ws:to> <ws:typ>month</ws:typ> </ws:ifx_sensor_request> </soapenv:body> </soapenv:envelope> 28

29 Dynamic Energy Profiles 1/3 Creation of the SPL stored procedure building_profile() Input parameter building_id (any level) from (DATETIME) to (DATETIME) aggregation granularity (DEFAULT = day. Can be hour, day, week, month, year ) Output parameter Building_id Timestamp Electricity consumption normalized in kwh Energy cost on peak Energy cost off peak Gas consumption (if available for that building level) CO2 emission CO2 emission scope 1 CO2 emission scope 2 Bed occupancy (only if aggregation granularity = day and building_id is a hospital) 29

30 Dynamic Energy Profiles 2/3 All aggregations are done dynamically against current data. No pre-aggregated results! Building has 140 spaces = 140 time series building_profile (" ", " :00:00", " :45:00", day") Avg runtime: 0.12 secs building_profile (" ", " :00:00", " :45:00", day") Avg runtime: 1.07 secs building_profile (" ", " :00:00", " :45:00", "month") Avg runtime: 11.1 secs building_profile (" ", " :00:00", " :45:00", week") Avg runtime: 10.9 secs building_profile (" ", " :00:00", " :45:00", year") Avg runtime: 10.8 secs building_profile (" ", " :00:00", " :45:00", year") Avg runtime: 21.3 secs 30

31 Dynamic Energy Profiles 3/3 Utilization of the built-in Informix time series functions to achieve the aggregation We are applying different time series calendar (variables on_peak_cal and off_peak_cal ) on-the-fly to calculate the on/off period consumptions based on the location of the building ( US or DE ). Variable report_cal is the target aggregation granularity. SELECT build_id, test.timestamp::datetime year to day date, test.value4::decimal(18,2), (NVL(test.value1,0.0)*on_peak_charge)::DECIMAL(18,2), ((NVL(test.value2,0.0) + NVL(test.value3,0.0))*off_peak_charge)::DECIMAL(18,2), test.value5::decimal(18,2), ((test.value5 * gasconvfactor * gas2kwh * co2factor) + (test.value4 * co2factor))::decimal(18,2), (test.value5 * gasconvfactor * gas2kwh * co2factor)::decimal(18,2), (test.value4 * co2factor)::decimal(18,2) FROM TABLE (TRANSPOSE (( SELECT UNION( AggregateBy('SUM($value)', report_cal, ApplyCalendar(r, on_peak_cal))::timeseries(performance_data1), AggregateBy('SUM($value)', report_cal, ApplyCalendar(r, off_peak_cal))::timeseries(performance_data1), AggregateBy('SUM($value)', report_cal, ApplyCalendar(r, 'cal_weekend_15min'))::timeseries(performance_data1), AggregateBy('SUM($value)', report_cal, r)::timeseries(performance_data1), AggregateBy('SUM($value)', report_cal, g)::timeseries(performance_data1) )::TimeSeries(performance_data5) FROM (SELECT Lag (TSRollup(reading, query_str1)::timeseries(performance_data1), lag) AS r FROM meters WHERE asset_id IN ( select asset_id from ifx_assets where building_id in ( select building_id from ifx_locations start with building_id = build_id connect by prior building_id = parent_id) and asset_type = "ELECTRICITY" )) AS r1, 31

32 Unit Conversions (PoC) Unit conversion implemented as stored procedures All kind of unit conversions, needed in this PoC, are based on fixed values. For every conversion a stored procedure (written in SPL language) was created. The procedures reside as precompiled statements within the database. Each part of the program can call the procedures and convert the units as requested. The following procedures have been created: - convert_btu2kwh and convert_kwh2btu to convert between BTU and kwh - convert_cbm2ccf and convert_ccf2cbm to convert between cft³ and m³ - convert_qm2sqft and convert_sqft2qm to convert between ft² and m² - convert_c2f and convert_f2c to convert between degree Fahrenheit and degree Celsius The conversion between local time and utc is implemented as procedure local2utcand utc2local. Daylight savings time in Europe and U.S. with different periods is implemented within these routines. The default output units are based on the location of the building. Units for display can be customised based on parameters in every script. 32

33 Agenda Background A typical Customer Profile Common Reasons to Act Anticipated Benefits The Solution Concept The Implementation Concept Informix Specifics Summary 33

34 Summary Cost reduction through better energy efficiency becomes more an more important these days There seems to be an attractive market for Smart Facility and Smart Building solutions/services out there A key component of a Smart Facility solution are sensor data Sensor data becomes only valuable if it can be related to relevant asset data (e.g. location of a sensor, room size) Informix in combination with or as part of an asset management system is a great foundation for a Smart Facility solution 34

35 Questions? 35

36 Alexander Koerner vcard 36