Network Design Recommendations Recommendations for s to take into account when doing network design to help create a more easily defendable and manageable network K-20 Network Engineering 6/30/15 Network Design Recommendation
Contents Network Goals... 2 Common Components... 2 Recommendations to protect and defend and analyze... 2 Example Network Designs... 5 Documentation... 8 Network Diagram... 8 Equipment Information Repository... 8 Network Run Book... 8 Communications Plans... 8 Conclusion... 9 For more information... 9 Resources... 9 Figure 1: Simple Network Design... 5 Figure 2: Multiple ISP Network Design... 6 Figure 3: SaaS Network Design... 7 K-20 Network Engineering 1
Network Goals In today s online world, services vital to organizations performing their day to day operations web presence, VoIP, Real Time Collaboration, outsourced payroll, Telemedicine, standardized tests are becoming more dependent upon network communications. Outages of many sorts looped ports, overloaded firewalls, saturated links, DDoS attacks can bring an organization to a screeching halt, as such specific design strategies should be deployed to enhance the survivability of services. This document aims to help create a more fully functional network which has the specific goals. Dependable Scalable Defendable Stable Manageable Common Components A network which is designed with the following components should be able to provide the functionality needed while keeping aligned with the aforementioned goals. Routers Switches VLANS network space Public network space Subnets Access Control Lists Demilitarized Zone (DMZ) Firewalls Network Address Translation (NAT) Log Collector(s) Traffic Policy Devices The proper implementation and use of these components will help to create a highly scalable, dependable, defendable and stable network that can help to simplify analyzing, managing and mitigating outages. Recommendations to protect and defend and analyze The following are a list of recommendations that organizations can institute to help defend themselves against becoming a victim of an attack, participant in an attack on another organization, or to help identify the flows involved in an attack. Hardware: For (s), use a which is able to easily process all the data flows expected for not only normal operations, but to include operations when in a failed or attacked state K-20 Network Engineering 2
For (s), use a which is able to provide some robust Access Control List (ACL) capabilities For stateful and stateless firewall(s), use a firewall which will have enough processing power to handle all data flowing through it and will should not buckle under extreme load For stateful firewall(s), use a firewall which is able to inspect all traffic and proxy all necessary protocols which may need translations help such as SIP, H.323, etc Use redundant network infrastructure configurations where possible Use managed switches Use switches which are compatible with your chosen Layer 2 loop elimination protocols Use switches which are able to perform storm control protocols Configure all hardware to export their logs to an external log collection server, in addition to on-device log stores All network segments: Where possible, try to create non-loopable Layer 2 (L2) segments Where L2 loop elimination is not possible, use L2 loop elimination technologies (STP, PVST, RSTP, TRILL, etc ) Use broadcast storm control technologies (BPDU guard, Stormcontrol, etc ) Monitor network segment for network performance metrics (throughput, errors, levels, etc ) Log performance statistics violations of network segment to logging servers (allowable bandwidth exceeded, Utilize smart log analyzers to detect suspicious activities (Fluentd, Sagan, Splunk, etc ) Set up easily modifiable rate limiting infrastructure Network Infrastructure Links Segregate network infrastructure Links to dedicated ports (not shared with other subnets) Use authentication schemes in accordance with dynamic routing protocols (OSPF, BGP, etc ) Mirror traffic to deep packet inspection engine (SNORT, Suricata, etc ) Log data flows to an analysis server (NTOP, nprobe, cflowd, StealthWatch, plixer, etc ) Monitor link utilization for historical, trending and real time statistics User Subnets Segregate user subnets to their own VLANS NAT user subnets with a many:1 private:public outbound Know and understand the type of traffic that should be traversing the user subnet Don t allow inbound connections from the internet to user subnets Utilize host firewall and anti-virus on all hosts in user subnet Utilize a firewall or Access Control Lists (ACL) to prevent inbound connections on the user subnets Use a stateful inspection firewall to protect OSI layers 4-7 from more complex attacks Server Subnets K-20 Network Engineering 3
Segregate private server subnets to their own VLANS Possibly NAT private server subnets with a many:1 private:public outbound Utilize host firewall and anti-virus on all servers in the private server subnet Filter access to services which should not be accessed remotely (RDP, fileserver, etc ) Utilize a firewall or ACL on the private server subnet to prevent any access into the private subnet from the internet Utilize a firewall or ACL on the private server subnets to prevent access to anything but the authorized services Public Server Subnets Segregate public server subnets to their own VLANS NAT server subnets with a 1:1 private:public space For stateful firewalls, utilize a firewall which is able to inspect all traffic and proxy all necessary protocols which may need translations help such as SIP, H.323, etc Know and understand the services running on the servers in the server subnets Utilize host firewall and anti-virus on all servers in the public server subnet Filter access to services which should not be accessed remotely (RDP, fileserver, etc ) VoIP Subnets Segregate VoIP communications infrastructure to their own VLANS Know and understand the services running on the servers in the VoIP subnets NAT VoIP subnets with a Many:1 private:public and only allow outbound connections Filter access to services which should not be accessed remotely (RDP, fileserver, etc ) Utilize a host firewall on all servers on VoIP subnets to only allow traffic to specific services DMZ Subnets Relocate high-target resources to a DMZ, if service off-site is not possible Utilize a host firewall on all DMZ servers to only allow traffic to specific services Filter access to services which should not be accessed remotely (RDP, fileserver, etc ) Filter access to services which should not be accessed remotely (RDP, fileserver, etc ) Offsite Subnets Off-site high-target resources (DNS, CRM, Web, Mail, etc ) to alternate service locations (colocation, SaaS, etc ) Use firewall and anti-virus as applicable on offsite services (SaaS, DaaS, IaaS, etc ) Monitoring Record all network events for analysis K-20 Network Engineering 4
Utilize smart log analyzers to detect suspicious activities (Fluentd, Sagan, Splunk, etc ) Utilize DDoS mitigation services to detect and clean dirty traffic Use IPS / IDS to prevent and detect intrusions Example Network Designs Below are some example network designs which incorporate a number of the proposed recommendations above to help create a more easily managed and defendable network which can lead to lessened impact by negative network events and ensure service availability. These designs are intended to be more logical designs with an understanding that there may be many instances of particular portions of the designs within an organization. While each of these designs focus on specific aspects of network design, parts of them can be combined and merged with each other to provide the appropriate network design that best fits the needs of the organization. FIGURE 1: SIMPLE NETWORK DESIGN Servers should be running local firewalls web server DNS Server collaboration server mail server dmz switch Public Subnet Via 1:1 NAT or Public addresses. Should have Firewall or ACLs on the Router limiting access to only the available services log collector monitor server file server Server Subnet On firewall or User Subnet On firewall or firewall Public and Network Space Internet Provider laptops PC As can be seen in the above diagram, the users and servers are segmented onto different VLANs to prevent local broadcast problems interfering with each other and taking out the entire network. The use of Layer 2 protocols such as spanning tree (STP, RSTP, PVSTP, etc ) and broadcast storm mitigation protocols (BPDU Guard, Storm Control, etc ) helps to prevent a looped port or an errantly broadcasting device from taking out all user and server subnets. Some services which may be high bandwidth or may not interact well with firewalling technologies are configured on a DMZ connected to the and are able to be protected with local firewalls and ACLs or firewall filters on the. The use of dynamic routing protocols such as OSPF may or may not be utilized in this K-20 Network Engineering 5
situation depending on the complexity and diameter of the network, for example if there were multiple s or multiple firewalls servicing all the network segments. The use of dynamic protocols helps to ensure that as new segments are added or new s are added that static routes do not have to be updated. Per user, subnet, or service rate limits may be placed on the firewall or to ensure that there are bounds on an errant devices or that may be trying to consume all the network resources. The and firewall in this scenario should be logging information about equipment performance, observed network events, user data flows, and any other information that is pertinent to the organization to the log collectors and monitoring servers so that real time stats and historical information can be reviewed for forecasting and forensic investigation. This option is a fairly common option for a medium sized organization which has a single internet connection and cannot afford to offsite services or purchase additional internet connections. FIGURE 2: MULTIPLE ISP NETWORK DESIGN Servers should be running local firewalls web server collaboration server dmz switch Public Subnet Via 1:1 NAT or Public addresses. Should have Firewall or ACLs on the Router limiting access to only the available services Internet Provider log collector Monitor server file server Server Subnet User Subnet firewall Public Network Internet Provider laptops PC Voice Subnet And only outbound Connections From Voice server allowed Phones Voice server While still implementing the monitoring and Layer 2 and Layer 3 protocols as referenced in the network design above, the above diagram has segregated services which may have been high likelihood targets or provide critical services to their own internet connectivity. This may be a separate connection coming into the organizations existing data center and utilizing VLANs and a K-20 Network Engineering 6
separate, or this may be a case where the organization has collocated the services to an external vendor. This prevents attacks to those high target servers from impacting the day to day operations of the users and local servers required for their day to day operations. It also provides a boundary so that issues which may arise within the local network are less likely to be able to negatively impact services which may be critical for business functions or brand identity. Also of note: in this scenario, the organization has deployed Voice over IP (VoIP) services, and have segregated those services onto their own VLAN to ensure that voice services are as protected as possible. This type of network design is quite often used by organizations which may have some of its web presence hosted externally. FIGURE 3: SAAS NETWORK DESIGN Servers should be running local firewalls web server DNS switch SaaS Provider log collector file server Server Subnet User Subnet firewall Public Network Internet Provider laptops PC As is the case with the previously mentioned network design, the design above has collocated some of its services to third party SaaS providers. This affords the organization the ability to have an external company manage and maintain the security policies and infrastructure for critical systems without taking on the complexity and cost of maintaining the appropriate security infrastructure. One other feature that is often offered by the SaaS providers is guarantees of uptime, specifically that 100% uptimes are guaranteed for critical services and brand identity. The cost of the SaaS services vary widely based upon throughput, number of servers required, levels of response, services being protected, etc While this design helps to ensure that services are always available, that does come at additional cost. K-20 Network Engineering 7
Documentation One of the most important things about managing a network is making sure that it is documented properly. Documentation of the network consists of a number different information stores, but a few types that are essential to ensure that all individuals are working with the same understanding and goals are listed below. Network Diagram All networks should have a detailed network diagram. Included above are some generic network diagrams, but the network diagrams that are generated for an organizations network should contain information about what servers and services are located on which systems and how they all interconnect. This should be in an easy to read pictorial format. This network diagram may be broken up into a number of smaller more detailed documents for clarity s sake, but there should be an overarching diagram which should reference detailed documents. There are a number of examples of what these diagrams may look at here in the link in the resources at the end of this document. Equipment Information Repository As part of the documentation for the network, there should be an equipment information repository. This information should be in an easy to review format and should be centrally located so all individuals of the organization know what equipment is active and the functions that equipment performs. This should also contain information such as warranty and support information. This information is operational information that may be referenced by the Network Run Book for appropriately responding to an incident within the network. Network Run Book Another key document to ensure proper network management and stability is the Network Run Book. This book defines all the processes that are utilized to manage and maintain the activities associated with the network. This includes functions such as adding in new segments, new servers, new s, new routes, new customers, new VPN connections, etc This book should be the authoritative guide for how to execute and react to incidents. This document should reference well defined communication plans, if there are to be communications about activities governed by this document. In addition to the Communications Plans for each of the activities in the Network Run Book, there should be a well-defined Roles and Responsibilities matrix developed. This should contain information about who is Responsible, Accountable, Contributing and Informed (RACI) about the procedures in the Network Run Book. Communications Plans Communications plans are vital for disseminating information about the going-on in a network. A well-defined and executed communication plan should be developed to ensure that when there are network issues that information can be broadcast and notifications can occur in a timely manner. The organization should have clearly defined roles and methodologies of communications, bearing in mind that normal means of communications may be hindered because of network unavailability, therefore non-traditional forms of communications, or old school forms may be more reliable. K-20 Network Engineering 8
Conclusion While this document is intended to present a number of recommendations for how to architect and implement a network, it is by no means authoritative in all circumstances, nor does it address every situation that may arise. The architecture and management of a network is an ever evolving process that changes with technologies, people, organizational needs, and any other number of factors. The main thing to be cognizant of is that the network should be constructed to be scalable, flexible, defendable, and manageable. For more information If you have more questions, comments or requests about the materials covered in this document or for additional documents, please contact the your organizations K-20 liaison, your sector representative, or the K-20 Program Office. If the matter is regarding an operational issue, please contact the K-20 NOC at noc@wa-k20.net or +1 (888) 934-5551. Resources Below are a list of resources which go into great detail regarding various aspects of the technologies mentioned in this document. These links are by no means the authoritative repository, they are meant to be a stepping off place for further research and evaluation. Here are some resources for some of the lower level components that should be utilized, these are very basic building blocks to help build up more complex networks. Router Switch Firewall VLANs http://www.webopedia.com/term/r/.html http://www.webopedia.com/term/s/switch.html http://www.webopedia.com/term/f/firewall.html http://www.cse.wustl.edu/~jain/cis788-97/ftp/virtual_lans/ http://serverfault.com/questions/188350/how-do-vlans-work http://www.webopedia.com/quick_ref/osi_layers.asp https://en.wikipedia.org/wiki/access_control_list http://creately.com/diagram-community/popular/t/network- OSI Model Access Control Lists (ACL) Network Diagram Examples diagram Network Run Book http://www.webopedia.com/term/r/run_book.html https://en.wikipedia.org/wiki/runbook Network Run Book Example https://contursiconsulting.com/documents/xyz_directory_runook.doc Below are some articles which go into comparisons of various logging and analysis tools with their strengths, weaknesses and caveats to them: Flow collection tools http://netflow.caligare.com/applications.htm Log Analyzer tools http://www.predictiveanalyticstoday.com/list-security-eventmanagement-log-analysis-software/ RACI Information https://en.wikipedia.org/wiki/responsibility_assignment_matrix K-20 Network Engineering 9