High Stability and Availability of emap System. Presented by: Amen Kwok

Transcription

1 High Stability and Availability of emap System Presented by: Amen Kwok

2 Teranet International leader and pioneer in the Electronic Land Registration Systems and Commerce. Exclusive provider of online property search and registration in Ontario. Own & operate The Property Registry system (TPR) in Manitoba. Other well known products like GeoWarehouse, ViMO, Purview, DoProcess, REDX, and Teranet- National Bank Housing Index.

3 Teranet Products

4 Teranet Industries In the Industry of Electronic Land Registration Systems and Commerce. Add value solutions also penetrate into the industries of Real Estate, Risk Management Solutions, Government & Utilities, and Law. Branham300's Top Canadian Technology Companies since Leadership award from URISA in the Geospatial industry for our commitment to Mapping and GIS technologies in 2012.

5 Teranet ESRI Users Headquarter in Toronto, Ontario and also located in Manitoba Employees located in Toronto as well as +125 employees in Manitoba office. Becomes ESRI users since 2011 when establishing emap system. Currently Acquiring the ArcGIS system for the Back Office map system.

6 emap Overview Ontario s Electronic Land Registration Map Rollout Was A Big Remark in Ontario Land Title & Map History. Provide Ontario Property Map & LRO Index Map. Feed online map to Teraview, POLARIS, ROSCO, LRO Admin, NGTV (Anticipated), and Consumer Channel (Anticipated). Covers broad range of customer base such as Ministry Staffs, Conveyancers, Paralegals, and LRO staffs.

7 emap Overview Cont d Processes to fetch updated data to emap from BackOffice Map Maintenance & Property Title Severance & Consolidation from LRO. 54 LRO index map are segregated. Presentation Tier: Central Map Viewer (CMV)

8 emap Downstream Flow Presentation Tier: CMV EMAP GeoDatabase

9 emap Upstream Flow Fetch property title changes to mark the outstanding parcel in EMAPin every 5 mins EMAP GeoDatabase Daily update process to fetch map maintenace data to EMAP Backoffice Map Database (MSE)

10 emap System Environment CMV Hardware: ESX server x2 Software: JBoss, ArcGIS Javascript API 3.7. emap Hardware: IBM x3650: 2 CPUs 4 cores 64 GB Mem x2 Software: EMAP: ArcGIS Server Linux Edition. GeoDatabase Hardware: IBM P7 server x 4 Software: Oracle Server Data Source: Teranet Back Office map data.

11 emap System Env Cont d Batch Process Software: FME 2012.

12 emap Cache Map

13 emap Dynamic Map

14 emap Print Map

15 emap Update & Outstanding Update Outstanding Map Update From POLARIS every 5 mins Daily Backoffice Mapping changes ownership parcels and full survey plans

16 emap High Stability & Availability emap system is targeted to operate in Extended Search Hours. Extended Search Hours (ESH): 4 hrs maintenance window in week days and 9 & 12 service hrs in Sat & Sun. Target to keep our availability in 99.7% in ESH. This requires high stability and availability. The system is anticipated to 7x24 service eventually.

17 Extended Search Hours Eastern Standard Time Monday to Thursday from 4:00 a.m. to midnight. Friday from 4:00 a.m. to 9:00 p.m. Saturday from 9:00 a.m. to 6:00 p.m. Sunday from 9:00 a.m. to 9:00 p.m.

18 Challenges ESH: 4 hour maintenance windows except weekend. Data Integrity: Data update from Back Office database during maintenace windows. Quality vs Performance: MapPlex being used in dynamic map. emap Requirement: Map view in dense area. emap Requirement: 54 LRO cached index map must be segregated. Corporate Standard: ArcGIS Server in Linux

19 emap Initial Design From the beginning, we planned to: Cost Efficiency: Host ArcGIS Server by 2 VM nodes. Requirement: 1 LRO Index Map / cache map service. Resource Efficiency: n Instance : 1 process. Default ArcGIS Benefit: Recycle the instances every 24 hrs. Conventional: Daily Update take traditional update process insert, delete, update.

20 emap Initial Design Cont d Requirement: 1 LRO Index Map / cache map service. Resource Efficiency: n Instance : 1 process. Default ArcGIS Benefit: Recycle the instances every 24 hrs. Better Map Quality: PNG24 format for export map. Conventional: Daily Update take traditional update process insert, delete, update.

21 emap Implementation Due to performance issue, plan changed. Performance: ArcGIS Server hosted by physical machines: 4 cores + 64 GB physical memory. Stability: 1 Instance : 1 ArcSOC in each map service. Avoid Memory Leak: Restart ArcGIS Servers weekly. Performance: Export map in png8 format, 96 dpi, & 1280 x768 max. Performance: Daily Update took Bulk Delete & Bulk Insert approach. Performance: 9 cache groups to serve segregated 54 LRO index map.

22 emap Cache Group

23 emap Cache Group Cont d

24 emap Cache Group Cont d

25 emap Implementation Cont d Stability & Avaliability: Strengthen Duplication & Failover in all tiers. Web Apps: Duplicated & Failover by Content Switch Presentation Tier: CMV loading balancing and failover in two nodes by content switch. GIS Tier: ArcGIS Server load balancing and failover in two nodes by content switch. Persistence Tier: Clustered in two node by Oracle RAC. Same structure hosting the replicated database standby in another data center.

26 Duplication & Failover MSS Data Centre Teraview/ ROSCO/ POLARIS MRK Data Centre Presentation Tier Content Switch CMV Web Server CMV Web Server GIS Tier Content Switch ArcGIS Server ArcGIS Server Persistence Tier Oracle DB Server Oracle DB Server Oracle DB Server Oracle DB Server EMAP GeoDatabase EMAP GeoDatabase

27 emap Implementation Cont d Stability & Availability: Full Monitoring & Performance Analysis. Presentation Tier - CMV: > LoadRunner script fetches synthetic transactions. > BMC BAC-BPM monitoring collect the responses time. > Report the performance and stats in Transaction Analysis. > Trigger alert if performance threshold exceeds. > SiteScope monitors the logs of CMV for any errors.

28 BPM Transaction Analysis

29 Full Monitoring & Performance Analysis Cont d Persistence Tier EMAP Oracle Database: > Patrol & BMC Knowledge Module monitors the availability of EMAP database. > Oracle Management Console will collect the stats and errors of EMAP database. GIS Tier: > SiteScope probes for each map service every 5 mins. > In-house built module to monitor for any error in log items of all map services and ArcGIS Server. > In-house built module to archive the logs and further produce the analysis report periodically to report summary of errors, transaction count, and any transaction longer than threshold.

30 Full Monitoring & Performance Analysis Cont d Full Monitoring, Stability & Performance Analysis. Batch Processes: > Patrol to monitor the service for outstanding property update which has been severed or consolidated but the map data has not been updated. > SiteScope monitors the logs of the batch process to update the EMAP data from the Back Office map data.

31 Monitoring, Stability & Performance Analysis Montior By: Perf by LoadRunner-BPM Log by Sitescope Presentation Tier Collect Stats: LoadRunner-BPM Montior By: ArcGIS Server by Sitescope + in-house monitoring module ArcGIS map services by Sitescope URL Monitoring Collect Stats: Analysis Report generated from ArcGIS log. GIS Tier Persistence Tier EMAP GeoDatabase Montior By: Partol Oracle KM Module Collect Performance Stats: Oracle Console Management. Daily Update Process Montior By: Log by Sitescope Outstanding Proeprty Update Service Montior By: Service by Patrol Log by Sitescope

32 The Project Highlights The emap implementation project has been planned since High involvement of ESRI consultation. The requirements of high availability and stability were evolved during the project implementation and after the initial rollout. We took more than 1 year to implement the addition for the high availability and stability continously. Production Support was also trained to handle any issue reported from any monitoring from various tiers.

33 Benefits From Using ArcGIS Produce the quality of map which emap was required. Low maintenance. Scalable through allocation of ArcSOCs to map services. Recycles its own instances periodically. Carries the performance info for each transaction in logs.

34 Lessons Learned High Stability & Availability should be planned at the very beginning. Use ArcGIS Server Windows Edition than Linux Edition. Use the native clustering of ArcGIS Server instead of content switch.

35 Future Plans Adopt the 3 rd party ArcGIS Server monitoring and performance analysis tool. (Support Linux Edition) Enhance the Log Analysis Report module to collect hourly transaction stats, chart production. Reduce cache groups from 9 to 5.

36 Future Plans Adopt Envision: How to link the log item among the user end application, presentation tier CMV, ArcGIS Server, and database. Health Check Automation. Anticipate to 7x24 service.

37 Q & A THANK YOU [email protected] Website: