Solving the Copper Replacement Conundrum

Transcription

1 Solving the Copper Replacement Conundrum Connecting Substations to the Network C. Harper Anderton President FAE Telecom

2 Contents Introduction... 1 Critical Network Infrastructure... 2 Solving the Conundrum... 4 Reliability... 5 The New Infrastructure... 5 Fiber... 5 Microwave... 6 Broadband over Power line (B-PLC)... 6 Carrier Metro Ethernet Services... 8 Conclusion... 9 All Purpose Transport Network Solving the Copper Replacement Conundrum - Whitepaper

3 Introduction Utilities and Pipeline Industries have relied on 4 wire analog circuits leased from the carriers to transport Supervisory, Control, and Data Acquisition (SCADA) information to and from remote terminals / substations (RTU) and provide the infrastructure for Pilot Wire protection for decades. These services offered an effective low cost, low bandwidth transport medium to enable basic monitoring services. These copper networks, using good old metallic cabling, long provided the bread and butter connection infrastructure for both business and consumer services. However over the last several years the carriers have viewed this infrastructure as a dinosaur, a relic of an age when voice was king and data was a word. Tier one carriers, such as ATT, Verizon, and CenturyLink, among others have announced plans to discontinue all copper based telephone services, starting within the next couple of years. Some services based on 2/4 wire lease lines have already been declared End of Life and dates to stop supporting them announced. Most major customers have been given end of service letters between now and Many Power Companies were sent letters telling them that one service they will lose is their lease lines which support SCADA access into their power substations some as early as Official filing have been made with the FCC as required by law notifying the Public of these end of life decisions by the carriers. This decision to cease support for the millions of miles of copper wire lines is the result of age, and the march of telephony to the universal use of wireless for both voice and data access. Today the carriers support two very different physical network plants. For the average person, how the phone network was constructed in the past and how it has evolved is something of a mystery. Throughout the first half of the 20th Century until around late 1950 s, all telephone calls were made on local copper wires, connected to a Local Exchange office which had a three digit prefix. Along with an associated area code. These six digits told the knowledgeable where in the country and in which city or state that central office was located. The final four numbers told you the entire number assigned to that home or physical address. The familiar xxxx would ring one phone number In the United States and Canada. That one number was assigned to a pair of wires from the central office for that address. The telephone network was built to allow any number to connect to any other number, by routing through the different Exchanges. This same concept applies to the TCP/IP Protocol. IP was derived from the original work and design of the United States Telephone network. As the population grew, more and more telephones were added resulting in more and more copper lines being run to support them. As this growth occurred the weight and cost of adding copper lines became overwhelming. Solving the Copper Replacement Conundrum - Whitepaper 1

4 Bell Labs scientists, then part of AT&T, and the Bell System began looking for solutions. They found them in electronics, called Multiplexers, and in use of Fiber Optic cable. By the late 1970 s this technology began to reduce the need for more copper wires. Network design, traffic planning for capacity, and many of the concepts for modern communication were invented at Bell Labs. Most if not all network architects and engineers, in the field of telecommunications, either were trained at Bell Labs, or by someone trained at the Bell Labs. In the late 1980 s cell phones were invented. Early models were heavy, but handsets got smaller and smaller and faster and faster. Today s smart phones have more processing power than the IBM370 mainframes of the early 1980 s. For many years it has been predicted that the rise of the Internet would push the use of computers and wireless technology past the capacity of the telephone network to support. All those dire predictions turned out to be incorrect. Capacity increases in the network, kept pace, as did constant improvement in the wireless spectrum as well. What happened in reality is wireless capacity has now reached the point where it is more cost efficient to maintain and grow the wireless backhaul than to add any more copper lines or to maintain the ones already installed. That simple economic fact, the rising cost to keep copper services working, and lowering the cost for wireless back haul explains the carriers reasons for their push to end support for their aging copper plant. As the carriers are now for profit businesses, they no longer are a protected and regulated business that must be maintained. Hence the discontinuance of the copper plant is an unstoppable force. The high cost of skilled technicians to keep the copper plant repaired and working and their technically skilled aging work force, have forced the carriers to take this approach. Critical Network Infrastructure For the vast majority of consumers and enterprise businesses this removal of services will not have a major impact. The number of homes who rely on a wireless phone is increasing every month and carriers are offering enterprise customers alternative services based on Ethernet and MPLS, leading to a steady decline in copper based services for the carriers. For most enterprise customers they have no alternative but to adapt to the new offerings as they are solely dependent on the carriers. 2 Solving the Copper Replacement Conundrum - Whitepaper

5 However for those customers whose applications require a TDM (Time Division Multiplexing) infrastructure to deliver guaranteed timing and latency for critical monitoring of remote locations, such as Power Utilities, these alternatives (wireless or MPLS) are not necessarily suitable solutions today. Much of the equipment being monitored is old and does not have an IP/Ethernet interface and the sheer cost of upgrading that equipment makes a simple transition impossible. The carriers lack of understanding or appreciation of the impact that the removal of these services will have is causing utilities considerable pain and leaving them no choice but to scramble to find workable alternatives. Within the power Industry, for those who have their own right of way, operate their own private networks, and have an increasing bandwidth need, there are options that will allow replacement infrastructure to be installed that allows for support of their current legacy (TDM) applications and provide an orderly controlled migration to the Next Generation Network that utilizes a packet based system. Solving the Copper Replacement Conundrum - Whitepaper 3

6 Solving the Conundrum As the current grid moves to the Smarter Grid and finally the Smart Grid, the telecommunications network infrastructure has to support all the new applications and subsequent increase in data that today s grid operators are deploying. A study conducted by Utilities Telecom Consulting Group on behalf of Tri- State Generation and Transmission Association, Inc. shows the bandwidth that is required at the various remote locations is increasing by as much as 350%. Application Data rate/volume Latency allowance Reliability Security (at end point ) (one-way) SCADA Medium/Low Low High High Smart metering Low/Very low High Medium High Operations data Medium/low Low High High Video surveillance High/Medium Medium High High Inter-site rapid response High/Low Very low Very high Very High (for example, Teleprotection) Distribution automaton Low/low Low High High Distributed energy Medium/low Low High High management and control (DER, storage, PEV) Mobile workforce (Push Low/low Low High High to Talk) Corporate data Medium/low Medium Medium Medium Corporate voice Low/very low Low High Medium Type of Facility Bandwidth Bandwidth Large Generating Station 1 DS3 1 OC3 ROC 1 DS3 1 OC3 Small Plant Combustion Turbine 4 DS1 8 DS1 Field Maint. Office/Warehouse 4 DS1 8 Ds1 230KV Sub Station, DC Tie 2 DS1 4 DS Kv Substation 1 DS1 2 DS1 Pole Top remote/switch 1 DS0 1 DS0 The TDM based networks used today provide guaranteed delivery, with a known latency, both for the primary link and also for any backup connection. 4 Solving the Copper Replacement Conundrum - Whitepaper

7 A key limitation of the copper services being used in the networks now is that the amount of bandwidth being provisioned cannot be increased. A new infrastructure has to be able to scale easily to allow the growth in bandwidth demand to be accommodated. Reliability The Goal is % (5 9s) up time in any critical network, no more than 5 minutes of outage in any 12 month period of time, which is achievable. The carrier networks were once built to this standard however they no longer expect to meet this level of reliability. They are best effort and if they meet a 95% uptime or availability mark, they consider that good enough. That can mean several hours of down time on a regular basis. Which may be good enough for dial tone, but is not acceptable for a Power Grid. People tolerate loss of their phones but get really upset when their power goes off at work or at home! Power Industry studies show that the lost productivity to the economic life of an area affected by an outage can never be fully made up. The reliability standard in the Power Grid control, monitoring and operations systems is now higher than the Carriers are able to deliver. This is one of the major reasons why considering a carrier based service for applications that affect Grid Control or substations monitoring may not be a wise choice for any utility. The New Infrastructure Any new infrastructure has to be able to address these key requirements. What then are the alternatives that allow us to solve the conundrum? 1. Fiber to all locations, including to the individual substation. Rings with subtended rings 2. Microwave links from a Fiber Backbone to the substations. A ring and string design 3. Broadband over Power line (B-PLC) using a ring and string design 4. Carrier Metro Ethernet Services Any of these approaches can be intermixed to optimize the cost of a specific use network. There is no one size fits all answer to a specific service area and application. Fiber In the best of all possible worlds with no limits for capital spending, an all fiber transport layer provides the most effective solution from a bandwidth growth, reliability and ease of use. However, it is the most expensive and hardest to justify. While fiber can be found as a primary transport on long distances and high voltage lines 248 KVA and up, this is not where the majority of substations are served. Today approximately 80% of substations in use today have either no connection or a narrowband connection and many cases no need for the amount of bandwidth a single pair of fiber can offer. Solving the Copper Replacement Conundrum - Whitepaper 5

8 Fiber construction cost varies widely across the United States. From a Low of $30,000 per mile in rural areas, to as high as $90,000 or more in urban regions of the USA. Stringing fiber to a new substation using OPGW (optical ground wire) is often the most effective way to deploy fiber, taking fiber to an existing substation can be both difficult and expensive and takes considerable time. Microwave As a matter of best practice the most proven and cost effective way to build any grid control and monitoring network has been a combination of fiber on the backbone, heavy traffic routes and microwave to the individual substation, a ring and string design. As a rule of thumb most of the planners and network engineers consider this the best choice if fiber is not a viable option to the substation However microwave has its own set of issues: 1. Geography, microwave requires unobstructed line of sight, this requires a full path engineering study to determine heights and path clearance. If line of sight is not available it can get very expensive, very quickly to ensure connections using licensed Radio. 2. Spectrum to license, is always an issue and must be coordinated with the FCC. Is licensed spectrum available? 3. Tower permitting, more and more localities are baulking at having tall microwave towers constructed in their territories. This is resulting in very long permitting cycles and in some cases making microwave untenable. 4. Unlicensed radio, while short distances and lower cost might be possible with unlicensed radio it is a near universal view of most Telecom Engineers in the power space, that unlicensed technology should never be used for mission critical path in the power grid, due to interference risk. 5. Cost, a typical microwave link installed between two locations can run from $50,000 to $Millions Broadband over Power line (B-PLC) Recent developments in Power Line Carrier technology has enabled a broadband communications connection to be delivered on power lines (up to 138KV), which covers the majority of the substation locations that currently use copper services. Broadband Power Line Carrier (B-PLC) is a possible game changer for the replacement of the copper based services being terminated by the carriers. Using the power lines that have to be present in every substation means no other approach can be applied as quickly or cost effectively to connect a substation into the backbone, making it an almost perfect alternative. B-PLC provides the ability to extend the fiber by connecting remote substations to the nearest substation on the fiber network using a Ring and String approach. As this is a new technology we are offering a more detailed examination of B-PLC here. The table below compares some of the key features of B-PLC to a copper leased line connection 6 Solving the Copper Replacement Conundrum - Whitepaper

9 B-PLC Features Powerline Communications Redundancy of multiple PLC phases and multiple transport technologies Low latency and low jitter network performance Standards based network End to end cyber security Utility Benefit Cost effective optimal use of utility s own assets. Leverages distribution and transmission power lines that are already being maintained. Significantly less expensive than stringing fiber. No line of sight issues or weather related performance impairments. Communications reaches wherever the line goes. High system availability for utility s most demanding critical applications. Supports network requirements of all utility s smart grid applications including real time protection and control. Interoperable with both TDM and IP equipment Utility is not locked to one vendor. Utility is protected and in full control of its own operational private network Solving the Copper Replacement Conundrum - Whitepaper 7

10 Plug and play serial protocol conversions. Supports legacy TDM, SONET and MPLS Scalable HV Coupling OFDM Rate Adaption B-PLC Differential Coupling B-PLC Frequency Agility Quality of Service (QoS) Integrates with legacy equipment providing utility with a migration path to a future smart grid network. Works on power lines up to 138KV without changing the electronics. Dynamically conditions the B-PLC transmitter in real-time to EMI disturbances on the power lines. Improves SNR by 10dB. Reduces EMI. Provides redundancy for single line faults. Allows for coexistence of multiple B-PLC channels when extending the reach of a single link and for multi-terminal applications. Priorities mission critical applications such as line protection. Carrier Metro Ethernet Services The carriers never like to give up a cash flow or revenue source, so yes the carriers do have their own offering as a replacement, an Ethernet based MPLS service. However this has a number of serious problematic issues: 1. Offering a managed Ethernet service muddies the waters considerably, as this is not an apples to apples comparison. Copper services, fiber, microwave and B-PLC are all effectively open pipes that are point to point and the utility manages 100%. The utility controls the flow of information and controls the prioritization of traffic. 2. The carriers MPLS service is not a point-to-point pipe that the utility can manage, it is a portal into the carriers cloud, and the only way they can offer traffic management is by layering MPLS on to this portal. This layering may not be compatible with the utilities network. 3. Much of the equipment being monitored and controlled in the substation is older legacy equipment that does not have any IP interface and it is just not feasible to upgrade all this equipment as well. 4. Whilst the utility can specify to the carrier the prioritization of the critical traffic generated within the substation, they have no control over the way the carrier prioritizes the utilities critical traffic through THEIR network. As this critical traffic passes through network nodes carrying other carrier s traffic, which utility has no control over. 5. How long will these services be offered? The carriers can terminate service offering at any time they wish (as we are seeing with the copper services). We have heard that some of the early Ethernet service offerings are now being terminated, with carriers forcing customers into higher bandwidth, more expensive offerings. Carrier based services are the least optimal way for a utility to build a critical network as it removes an area of network infrastructure from their direct control and responsibility and puts them at the mercy of an outside service organization that may not share their priorities. 8 Solving the Copper Replacement Conundrum - Whitepaper

11 Conclusion The announcements from the carriers that the copper based services were to be discontinued has created consternation and worry amongst the utility companies. The enormity of the transition in a relatively short time frame has resulted in the Utilities Telecom Council (UTC) setting up a working group to help their members with the decisions involved. There are alternative infrastructure solutions available today, each with its benefits and limitations as shown in the table below: Feature Carrier Copper Service Microwave Fiber Broadband PLC Line protection (POTT/DCB) Yes yes yes yes Current Differential yes no yes yes SCADA yes yes yes yes Backhaul Communications no yes yes yes Broadband no yes yes yes Data Rate Kbps Mbps Gbps Mbps Utilizes existing line no no no yes Time to deploy medium long long short Service disruption medium low high low Cost effectiveness medium medium* low high * Assumes licensed radio FAE Telecom s position is that there is no one single answer, no one size fits all solution. Each substation connection will need to be looked at individually and the ideal connection selected based on the factors discussed in this whitepaper. We see fiber as being the primary choice where it s high cost can be justified. If fiber is not an option then the choice is between B-PLC and microwave, with geography, distance, and installation time all being critical factors. The one solution we see as being the solution of last resort is another carrier service. The lack of understanding and appreciation of the utilities requirements and the carriers current business model make them a very unstable partner for building a critical monitoring and control network. Solving the Copper Replacement Conundrum - Whitepaper 9

12 All Purpose Transport Network The key issue with the replacement strategy is adopting a migration strategy that allows the replacement infrastructure to support the TDM based applications that are critical to monitoring the network today and that allows a managed organized migration to the packet based applications that are the future. The three key alternatives discussed in this whitepaper fiber, microwave and B-PLC all allow that migration. They are pipes that the utilities own and control and will carry any type of communications traffic. The migration path has to be what we call an APTN, All Purpose Transport Network. This is a multifaceted, multi-component infrastructure that allows for diverse application support. At the edge will be multiple access infrastructures, fiber, B-PLC and microwave. In the regional network it will be fiber (SONET/COE) or microwave and in the core fiber (SONET/COE). Running over this infrastructure will be multiple applications with traffic management at the edge being TDM, COE, MPLS-TP or a mix. At the regional network this could be COE, MPLS-TP of MPLS or again a mix. The core network will be high capacity fiber with a COE/MPLS mix. PO Box 2842 Acton, MA info@faetelecom.com 855.GO-FAETEL 10 Solving the Copper Replacement Conundrum - Whitepaper