Challenges in Today s Enterprise Network Operations

Transcription

1 Challenges in Today s Enterprise Network Operations Perfect days are rare for today s enterprise networking professionals, but modern monitoring tools provide visibility to get the job done. Sponsored by ebook: Challenges in Today s Enterprise Network Operations

2 Life in a Network Operations Center: Where Our Digital World Actually Happens In the Internet age, there are few jobs more important, in terms of keeping our world running smoothly, than those held by the men and women maintaining large-scale enterprise networks. Most people outside of technology circles know little or nothing about what it is these network operators and network engineers latter-day sentries in an increasingly connected world really do, or how modern technology in the form of cutting-edge network monitoring equipment provides a consistent, stable platform from which they keep watch over systems critical to large enterprises and individuals alike. If not for the efforts of network professionals and reliable networking solutions, enterprises would run more expensively and less efficiently, and we d all face more frustration in banking, shopping or any other 21st century endeavor requiring Internet interaction. Over the course of the following pages, we ll cover the various challenges affecting today s network operations center, from the multitude of information streams that need to be constantly monitored to remote data centers that need managing from afar. We ll attempt to pull back the curtain on this high-pressure, high-stakes world occupied by engineers who design networks and solve tough problems and operators charged with monitoring and troubleshooting real-time issues that arise in a production environment. They occupy the hot seat day-to-day, working to tame Big Data and combat constant assaults from cybercriminals. And, we ll explore new tools that make their work more efficient and effective. ebook: Challenges in Today s Enterprise Network Operations 2

3 A Life Under Constant Pressure Generally speaking, the action in the data center is divided between network operators and engineers. Operators work in high-tech control rooms surrounded by computer monitors, a telephone, and perhaps several big screens displaying either live video of, or performance data about, their companies data centers. These are the kind of places that few people are ever allowed to enter, in part because of data security concerns. Although they are removed from production data transfer, NOCs monitor and adjust the channels through which general data traffic passes. Government secrets, highly confidential financial data, medical records everything happening on public-facing networks is controlled from these rooms. NOCs are the nerve centers of the connected world. Network engineers oversee all these digital connections between people and corporations. Their job is to make sure the information keeps flowing, and that servers aren t overwhelmed. And, if anything misfires (as it inevitably will), it s up to these guys to identify, diagnose and fix the problem so fast that few people out in the world are affected or even notice. There are few real-time action centers like a NOC in the technology world. When an entire enterprise network is hanging in the balance, it s not uncommon for a NOC operator to work under tremendous pressure from above. ebook: Challenges in Today s Enterprise Network Operations 3

4 What a NOC is Like: Rarely a Perfect Day On a perfect day, network operators sit at their control station, quietly monitoring anywhere from a handful to dozens of screens that spit out information about the status of the equipment through which dozens or even hundreds of gigabits of information continuously flow. Upon arriving for work, they usually are briefed by the operators who are about to end their shifts regarding any ongoing or recurrent problems the network is experiencing, and given a list of changes, updates and tweaks to the network and various devices on the network that need to be accomplished during their shift, if possible. The newly arrived operator then makes a quick assessment of the network s status before turning to that list of more-or-less ordinary, daily tasks. But perfect days are rare. A network operations center typically has many fires burning simultaneously, says Dan Dowiasz, Network Solutions Engineer at APCON, a leading maker of sophisticated network switching equipment used to manage and secure the high-speed flow of data. It s a multi-faceted role. Network operators must have a broad understanding of companies entire networks and technical operations so that they can quickly spot problems, properly diagnose them, and address them effectively and efficiently. And while they may possess deep expertise in various aspects of their networks, their job is to take a pretty holistic view of the [network] environment, Dowiasz says. They have the entire network [to monitor]. They re going to be looking at specific routers. They re going to be looking at WAN connectivity. And they re going to be looking at individual applications in some cases. ebook: Challenges in Today s Enterprise Network Operations 4

5 Juggling Skills are Premium In This Environment The biggest challenge in the NOC, Dowiasz says, is prioritizing and deciding how to address issues that invariably crop up in even the most sophisticated, state-of-the-art data and communications networks. Then they have to quickly bring (those issues) to some level of restoration, or escalate them, he says, to a second tier of technology trouble-shooters who have the expertise and time to dive deeply into particularly vexing problems. Typically, network operators have a defined time limit to diagnose and remedy problems they uncover in the network on their own (or network problems to which they re alerted when users call in for help). If they can t complete the diagnosis and repair process within those time limits, most companies operating procedures require them to escalate the issue; to pass it up the second-tier troubleshooters. In nearly all cases, time equals money when it comes to keeping the network humming. Uptime is all-important. Outages or slowdowns can quickly cost millions. And, oversubscribed servers or monitoring tools can easily lead to dropped data, meaning companies lose the complete visibility they need to ensure data integrity and/or security. As we said, the stakes can be very high. How quickly did you identify and resolve that problem? is the central question network operators typically must answer to their bosses satisfaction. The answer they can give might well depend on the network architecture and whether or not it includes an intelligent network monitoring switch that gives them 100-percent visibility into all traffic. Total visibility is the key to making solid, real-time decisions that keep networks humming and bosses happy. ebook: Challenges in Today s Enterprise Network Operations 5

6 So Little Time, So Much to Monitor. In old Western movies, there was often a character who dropped to his knees to put his ear to the ground or a railroad track during a chase scene. The goal was to try and hear whether riders or a train, though still out-of-sight, might be approaching. Whether such practices ever were really used is shrouded in folklore. But they might represent an early effort at using tools to monitor a network. The job of today s network operations professionals is vastly more complex, requiring many more ears to the ground, so to speak. The network monitoring and switching tools are vastly more sophisticated as well and they need to be. On an atomic level, consider network access to a single database. Data flows in from an outside point, through a firewall that is tapped on either side. Then you get to a distribution router, which again is tapped on either side. Then the information is sent into one part or another of the data center. Maybe there are three or four inputs to that data center, each tapped on either side. And so on. That s a lot of points to monitor, and in some scenarios it could multiply significantly. If the network operator has only so many copies of expensive monitoring tools for data recording or security, for example then he is forced to move it around, either physically on a crash cart or logically. That process, is not only a pain in the neck. It also does not provide complete visibility. That s why a rock-solid monitoring architecture is absolutely central for a NOC and why intelligent network monitoring switches, which aggregate all that information so tools can achieve 100-percent visibility, provide the heartbeat of daily activity. Without that ability, operators will have a tougher time knowing, for example, that tools are close to being overloaded, or that bandwidth usage is approaching its limit for critical network devices, or if a network is being intruded by an unwanted DDOS attack. ebook: Challenges in Today s Enterprise Network Operations 6

7 The Other Issue: How to See What s Happening Another big issue is how to give operators the ability to easily see the metrics that indicate whether data is flowing at the proper speed, to the proper destinations, and in the proper form. In days gone by, monitoring and switching equipment displays would be filled with rows of numbers. Redirecting data on the fly meant deciphering and rewriting dozens or hundreds of command lines. Today, an uncluttered graphical user interface ideally a web-based interface can be a NOC operator s best friend. Many of those key indicators can be brought together under one pane of glass so operators can monitor several variables without several browser windows or screens. Intuitive, easy-to-read and highly-reliable GUIs now allow a single NOC operator, or a pair of them working together, to monitor and effectively manage literally hundreds of listening points across their networks and respond quickly and accurately in diagnosing and solving problems, all while handling day-to-day updates and network maintenance tasks. NOC operators usually can project a dozen or more dashboards onto their displays. Each dashboard provides the operator a quick, easy-to-decipher, and usually color-coded readout of essential system or network performance data. These typically are presented as numbers and/or graphs that make it easy for operators to quickly process their meaning. ebook: Challenges in Today s Enterprise Network Operations 7

8 The Other Issue: How to See What s Happening When sensors detect a problem anywhere on the network, or in any of the analytical tools and switches tied to the network, a GUI-controlled dashboard typically lights up, changes colors and/ or pushes other dashboards with which it normally shares space off the screen. In some cases, audible alarms can be set to ring to make sure the operator s attention is drawn to the issue. Once alerted to the existence of a problem, the operator then can drill down on it by going one or several screens deeper into the GUI to locate and diagnose the specific problem. Then he or she can enter simple command, usually in his or her native language commands rather than entering a long string of complex code in order to resolve the problem. ebook: Challenges in Today s Enterprise Network Operations 8

9 Managing Lights Out Operations The evolution of network management tools and tactics have not only made it possible to more effectively and cost efficiently manage networks with smaller staffs. It also now makes it possible to manage multiple remote data centers from one central control facility. Even totally lights-out, or dark, data centers, where no human is actually assigned to work, can be operated remotely if companies make wise decisions to provide remote data centers with the right network monitoring switches and connectivity before problems even arise. The savings possible with remote operation of lights-out data centers are considerable. For years, one of the ways companies managed data center operating expenses has been by sharing troubleshooting and even some analytic and data-flow management tools. When problems were detected, technicians would physically move analytical tools, or sniffers, designed to diagnose or repair the problem, from one rack of servers and components to another. Doing so was never easy. It required a technician to wait until the tool being moved completed the work it was already doing. Then he had to physically move it on a cart, which was pushed across the room or in some cases to another floor or building. Additionally, the technician typically had to move cables from one location to another, properly reconnect the tool to the network and, in some cases test and recalibrate the tool before it could be used. Meanwhile, the original problem detected by the NOC remained undiagnosed and unfixed. ebook: Challenges in Today s Enterprise Network Operations 9

10 Managing lights-out operations can get out of hand If a data center is dark and remotely monitored and controlled, one of the big challenges comes when an outage happens or a problem is detected and the proper equipment is not in place to remotely troubleshoot the issues. If they have to do some re-cabling and an equipment change they re out of luck, says Ronnie Sun, a field network architect who works with APCON customers in designing their networks and data centers to achieve maximum utility of switches and the various analytic tools used to monitor traffic. It can take all day, or even longer to move someone there and get the necessary changes made. And the cost in time, travel and disrupted network operations can be quite high if technicians have be dispatched to remote data centers that are tens, hundreds or even thousands of miles away, or if local contractors have to be engaged to go in do work at what typically would be well above-market rates of pay. That s why, until recently, many companies had been reluctant to switch to managing some data centers remotely. ebook: Challenges in Today s Enterprise Network Operations 10

11 Today s Remote Monitoring Capabilities Streamline the Process, Reap Savings Today, however, advances in intelligent network monitoring switches and analytical tools and software are making it possible to allow companies to provide remote management of data centers in a cost effective way. The latest remote monitoring switch technology represents a costeffective approach for any size enterprise. Thus, the NOC operator on duty at a company s primary data center on the West Coast can effectively troubleshoot, diagnose and fix problems at a remote data center the same as if the problem were in a server right down the hall from them. Problems can easily be diagnosed and resources re-provisioned from a single desktop screen or even from a mobile device when the manager checks in from home or the road. In fact, by taking advantage of new multi-port and higher-bandwidth technology, network problems or outages can now often be addressed faster and at lower cost via remote control that they were just a few years ago when technicians physically staffed those now-remote data centers. ebook: Challenges in Today s Enterprise Network Operations 11

12 Multi-Switch Management Saves Effort, Money in Network Operations The value of managing multiple switches from one place goes well beyond the NOC and all the way to the C-Suite, which can save considerably by applying the power of pricey analytics tools across multiple locations. In the not-too-distant past, the typical corporate technology budget manager sought to skimp on monitoring tools, sniffers and the like. He figured that the cost and hassle of carting tools between different servers and nodes on an as-needed basis was offset by the savings from not buying relatively expensive tools and the pricey, dedicated switches that each such tool typically required. As a result, enterprises sacrificed visibility. But the advent of scalable network monitoring switches that can flawlessly duplicate, slice, repackage and/or deduplicate data streams and redirect those streams to a large number of analytical, monitoring and security tools all without the creation of excessive data latency has changed the financial math. ebook: Challenges in Today s Enterprise Network Operations 12

13 IT managers need ways to cope with data demands Likewise, the rapidly rising need to increase corporate networks bandwidths from 1G to 10G and 40G network speeds is pressuring IT managers to look for new switching solutions that can handle the increased data flow demands with minimum dropped packets and high availability. They need to quickly and systematically sift through these loads and filter for information of interest. In addition, they are also under pressure to reduce IT spending in general, and data center spending in particular. The hidden beauty of scalable switches is that, while they cost more than fixed-size switches, each scalable switch rewards the buyer with economies of scale. A single scalable switch can stand in the place of a dozen or more fixed-size switches. IT budget managers can do well by replacing existing conventional switches with a lower number of scalable switches. Similar economies of scale are produced by feeding various analytic, monitoring and security tools from the same monitoring switch instead of linking each tool to a dedicated conventional switch. There s also a degree of backup capacity associated with most monitoring switches. Today s cutting-edge intelligent network monitoring switch can offer up to 288 1G or 10G ports in one chassis. In large enterprises, it s not uncommon to employ several dozen ports. It is in such cases that today s NOC can be most effective, and all the advanced tools work together like a symphony. If all goes well in the NOC operator s day, it all happens so fast that a customer just logging in to check his bank balance might never notice an interruption in the flow of data. That NOC operator might just have made that customer s day, and he ll never know it. ebook: Challenges in Today s Enterprise Network Operations 13

14 APCON, Inc SW Pioneer Court Wilsonville, Oregon USA Tel: Toll Free: Engineering Design Center 501 W President George Bush Highway, Suite 100 Richardson, Texas USA sales@apcon.com APCON, ebook: Inc. Challenges apcon.com in Today s +1 Enterprise Network Operations APCON, Inc. All Rights company/apcon APCON is an Equal Opportunity Employer MFDV R1-0314