AltaLink IP/MPLS Network Nov 3, 2011 Stephen VanderZande
Contents 1. Introduction 2. Current Network 3. Why IP/MPLS? 4. What is IP/MPLS? 5. Services 6. AltaLink s Project 7. Test Results (time permitting) 2
Introduction Stephen VanderZande Electrical Engineer NetCom, AltaLink Configuration, Design, and Implementation Operation and Troubleshooting Clinton Struth Principal Engineer - NetCom, AltaLink Cory Struth IP Network Architect NetCom, AltaLink 3
Current Network Existing TDM/ATM network. Product discontinued and support for technology decreasing. Provides TPR, SCADA, OPX, Mobile, and IP Services. Increasing demand for IP services (data collection/security). Centrally managed. Critical to have a scalable network. Assigned paths and traffic prioritization. Connection based services. Traffic Engineering 4
Why IP/MPLS? Why now? Existing ATM / TDM equipment has been discontinued. Increasing demand for IP from new and migrating services. Need to support legacy services and protocols. MPLS can support legacy services (RS232/G.703*) and is protocol indifferent. Transitioning to packet-based transport (Ethernet radio and fiber). Need to deploy to all sites - >300+ nodes MPLS used heavily by telecom carriers. 5
Why IP/MPLS? Why now? Traffic Engineering Defined traffic paths and redundancy. Enhanced Quality Of Service(QOS) Traffic Prioritization. Advanced IP Services VPLS, VPRN Bandwidth Flexibility Enhanced centralized management and service provisioning. Separation of the control plane and the data plane. Secure IP traffic. 6
What is IP/MPLS? An IP/MPLS network is a packet-switched network that uses Internet Protocol (IP) enhanced with Multi-Protocol Label Switching (MPLS). IP Resilient but best effort delivery of packets. Delivery times are not guaranteed and congestion can be a concern. IP traffic is delivered based on its destination IP address. Traffic Engineering and QOS is not scalable. 7
What is IP/MPLS? An IP/MPLS network is a packet-switched network that uses Internet Protocol (IP) enhanced with Multi-Protocol Label Switching (MPLS). MPLS Provides connection-oriented services and is protocol independent. Provides traffic delivery using pre-specified primary and backup paths (Traffic Engineering). More predictable delivery times. Scalable QOS provides traffic prioritization. Uses labels to move traffic faster and with less processing. Supports advanced IP services (VPLS, VPRN). Increased security (separation between data plane and control plane). 8
Services 9
Services - Standard SCADA Substation to control center communication used to monitor and control the transmission system. RS232, DNP3 over IP Teleprotection (TPR) Relay to relay communication used to speed up fault clearing. Analog, RS232, G.703, C37.94, IP Voice (OPX) and Mobile Substation phone communication and mobile radio support. Analog, VoIP Substation LNA Used to collect data from relays, meters, condition monitors, and all other IP capable devices in the substation. Also provides field network access and internet. Ethernet, IP *** Future Protocol Possibilities *** 10
Services Advanced IP Services Virtual Lease Line (VLL) Virtual Private LAN Switch (VPLS) Virtual Private Routed Network (VPRN) Network as seen by end-user. 11
Services Advanced IP Services VPRNs and VPLSs allow secure separation of IP services. Allows service specific prioritization and restrictions (QOS). Example Services: Security Video Field Internet Access Corporate Network Access Synchronization (1588 Timing) Radio Monitoring Substation LAN External Backhaul 12
AltaLink s Project Completed Calgary Core Build 2010 (red) Tomahawk Spur Pilot - 2010 (purple). 2011 deployment to upgrade backbone Oc3 network + select sites (green). 2012 plan Start upgrading substation sites in mass over 100 sites each year for 2012 and 2013. Many sites to be upgraded in parallel as part of major projects, capital maintenance projects, etc. 13
AltaLink s Project 2010 Tomahawk Spur LAN, SCADA, and TPR services moved to MPLS. TPR service latency below 10ms one-way (138kV). Firewalls installed for LAN traffic. Reliability has been as-good or better than TDM equipment. Identified previously unknown performance issues with radios. 2011 OC3 Backbone MPLS emulating legacy ATM network. OC3 ring cutover completed Sept 22 nd. Working out the bugs and optimizing setup of our hybrid network. Again identified previously unknown performance issues with radios. 14
Questions? Stephen VanderZande stephen.vanderzande@altalink.ca 15
Field Test Results Case 4 Case 5 Case 6 Case 7 PLS (octet) 16 8 8 16 JBS (ms) 10 10 5 5 Service Round trip incl. tester G.703 (64kbps) (from T1 ASAP) from 3600 19.234 19.724 13.887 15.708 RS-232(9600 baud) from SDI 28.968 29.423 23.392 25.288 RS-232(19200bps) from SDI 23.048 23.661 17.295 19.557 RS-232(38400bps) from SDI 20.422 20.489 15.321 16.677 Analog (DS0) from T1 ASAP from 3600 22.96 24.64 15.32 16.68 One Way exl. Tester (0.374ms) G.703 (64kbps) (from T1 ASAP) from 3600 9.617 9.862 6.9435 7.854 RS-232(9600 baud) from SDI 14.484 14.7115 11.696 12.644 RS-232(19200bps) from SDI 11.524 11.8305 8.6475 9.7785 RS-232(38400bps) from SDI 10.211 10.2445 7.6605 8.3385 Analog (DS0) from T1 ASAP from 3600 11.48 12.32 7.66 8.34 Values < 10ms one-way = green Objective was to have one-way latency for most services <10ms. Only 9600bps RS-232 couldn t get below 10ms objective. SReXperts EMEA 2011 - Warsaw Poland 16
TDM / ATM Baseline (Current State) TDM-only circuit (straight DS0 cross-connect). Latency = 1.563ms (round-trip) TDM+ATM DS0 cross-connect + ATM circuit (VPI/VCI) Latency = 9.312ms (round-trip) SReXperts EMEA 2011 - Warsaw Poland 17
MPLS OC3 and Gig-E Transport Add in MPLS transport layer using E-pipe service + OC3 transport. Latency = 11.172ms (roundtrip) Replace OC3 with Gig-E (packet transport or fiber) Latency = 11.17ms (round-trip) [virtually unchanged] SReXperts EMEA 2011 - Warsaw Poland 18
Teleprotection Field Pilots Two transmission lines four relay schemes (A&B per line). Wolf Creek Radio 9543R 4T1 Pinedale 207S 890L A-channel = SEL 311C (RS-232 Distance) 28T1 890L B-channel = GE D60 (G.703 Distance) Bickerdike 39S 740L A-channel = GE L90 (G.703 Differential) 740L B-channel = SEL 311L (G.703 Differential) 740L 4T1 Edson 58S 890L SReXperts EMEA 2011 - Warsaw Poland 19
890L Distance Relay Scheme (Permissive) 890L A-channel = SEL 311C (RS- 232) RS-232 C-pipe Service 38400bps, async Jitter buffer = 5ms Payload = 16 bytes Circuit built with: Strict-hops (RSVP-TE) FRR Lab results = 5.9ms (one-way) Actual results = 7.185ms (oneway) 890L B-channel = GE D60 (G.703) G.703 C-pipe Service Use 3600 to convert G.703 to TDM timeslot, map timeslot across MPLS. Dual-circuit relay (two circuits for one relay) PL/JB = 16/10 Circuit built with: Strict hops (RSVP-TE) for both go and return paths (to ensure symmetry) FRR Lab results = 7.17ms (one-way) Actual results = 8.475ms (one-way) SReXperts EMEA 2011 - Warsaw Poland 20