SECURING THE S T A C K WEB NETWORK OPERATING SYSTEM MEHUL SHARMA BOSTON UNIVERSITY

SECURING THE S T A C K WEB NETWORK OPERATING SYSTEM MEHUL SHARMA BOSTON UNIVERSITY

C A V E A T S

C A V E A T S Everything is based purely on linux -- no outside vendor / 3rd party software Standard kernel, iptables, posix acls, routing stack and bridge and related packages which means you can get support from the distro provider or upstream / mainstream kernel mailing lists. I am not a guru and definitely not good enough to be a student. so keep enough room for mistakes, and don t believe me if you don t want to. I am attempting to show you what is possible from my experience building & deploying appliances from high performance commodity hardware using my intellectual properties. If done right, it will change your perspective, not to mention reduce your TCO exponentially and give you extreme flexibility and performance -- you will not be enslaved any more.

TOPICS OF INTEREST

Substantially Reducing to Stopping Denial of Service Attacks & Data Mining towards the Web Infrastructure in an Automated Manner Building a Distributed, Secure, Ad-hoc & Automatic Converged Network System Posix ACLs for Heightened Compliance Control over Operating System Infrastructure

SECURING THE S T A C K W E B S E C U R I T Y

Aim of Accomplishment Reduce-to-Stop Flooding from a Single Connection / Flow Not allow more than a certain number of connections from a single source IP address Not allow a source IP address or a range of source IP addresses to use more than a defined amount of data transactions to part or parts of the web infrastructure Look for Data Mining attempts in an intelligent manner by looking only at PSH+ACK packets and take necessary action(s). Everything achieved by IP/EBTABLES at Layer 2 or Layer 3 or their combination

SECURING THE S T A C K -- WEB SECURITY Reduce-to-stop flooding from single connection / flow into FLOODGATED Restrict a defined number of connections from a single soure IP F L O O D G A T E S

SECURING THE S T A C K -- WEB SECURITY Check for aggresive IP behavior and send it to INUNDATED to be blocked & logged. If not, move on to MINERGATES to check for data miners C H A N N E L

SECURING THE S T A C K -- WEB SECURITY Check for data mining activity and send to MINERS for resetting TCP connection & logging. M I N E R G A T E S

SECURING THE S T A C K -- WEB SECURITY Give selected IP address ranges pre-defined bandwidth quotas B A N D W I D T H G A T E S F U L L H I G H M E D I U M L O W

SECURING THE S T A C K -- WEB SECURITY # Clean all rules and chains iptables -F iptables -X # Create the following chains: iptables -N FLOODGATES iptables -N CHANNEL iptables -N INUNDATED iptables -N MINERGATES iptables -N MINERS iptables -N BANDWIDTHGATES ## Drop fragmented, xmas and null packets iptables -A INPUT -f -j DROP iptables -A INPUT -p tcp --tcp-flags ALL ALL -j DROP iptables -A INPUT -p tcp --tcp-flags ALL NONE -j DROP ## Create a FROZEN & MINER_FREEZE list - viewable via /proc/net/xt_recent/ iptables -A INPUT -p tcp -m tcp --dport 80 -m recent --update --seconds 2000 --name FROZEN --rsource -j DROP iptables -A INPUT -p tcp -m tcp --dport 80 -m recent --update --seconds 3000 --name MINER_FREEZE --rsource -j REJECT --reject-with tcp-reset ## Pass all TCP packets through FLOODGATES iptables -A INPUT -p tcp -m tcp --dport 80 -j FLOODGATES

SECURING THE S T A C K -- WEB SECURITY ######## FLOODGATES ######## ## Limit the pre-defined number of connections from a single source IP and reject the rest of them with TCP reset iptables -A FLOODGATES -p tcp --syn --dport 80 -m connlimit --connlimit-above 5 --connlimit-mask 32 -j REJECT -- reject-with tcp-reset ## Check for the rate of defined TCP SYN packets per second and drop & block them for a pre-defined time if above the limit ## viewable via /proc/net/ipt_hashlimit/ iptables -A FLOODGATES -p tcp --syn -m hashlimit --hashlimit 10/sec --hashlimit-burst 3 --hashlimit-htable-expire 36000 --hashlimit-mode srcip --hashlimit-name floodgated -j ACCEPT iptables -A FLOODGATES -j CHANNEL ######## CHANNEL ######## iptables -A CHANNEL -m recent --set --name AGGRESSION_CHECK ### Source IP matching 12 times in 100 seconds - send to INUNDATED iptables -A CHANNEL -m recent --update --seconds 100 --hitcount 12 --name AGGRESSION_CHECK --rsource -j INUNDATED ### Source IP matching 10 times in 20 seconds - send to INUNDATED iptables -A CHANNEL -m recent --update --seconds 20 --hitcount 10 --name AGGRESSION_CHECK --rsource -j INUNDATED iptables -A CHANNEL -p tcp -j MINERGATES

SECURING THE S T A C K -- WEB SECURITY ######## MINERGATES ######## ## Look for 3 way handshake -- # Client connects to the server by sending a SYN (Synchronization) packet. # Server responds with a SYN + ACK (Synchronize + Acknowledgment) packet. # Client responds with an ACK (Acknowledgment) packet and communication is is established. # Client sends its first PSH + ACK (Push + Acknowledgment) packet which contains the http header. iptables -A MINERGATES -m recent --set --name MINER_CHECK iptables -A MINERGATES -m recent -p tcp --tcp-flags PSH,SYN,ACK SYN,ACK --sport 80 --update iptables -A MINERGATES -m recent -p tcp --tcp-flags PSH,SYN,ACK ACK --dport 80 --update iptables -A MINERGATES -m recent -p tcp --tcp-flags PSH,ACK PSH,ACK --dport 80 --update --seconds 10 --hitcount 2 --name MINER_CHECK -m string --to 70 --algo bm --string "GET /search/?username-directory=1&query=" -j MINERS iptables -A MINERGATES -m recent -p tcp --tcp-flags PSH,ACK PSH,ACK --dport 80 --update --seconds 60 --hitcount 22 --name MINER_CHECK -m string --to 70 --algo bm --string "GET /index.html" -j MINERS iptables -A MINERGATES -p tcp -m tcp --dport 80 -j BANDWIDTHGATES

SECURING THE S T A C K -- WEB SECURITY #### INUNDATED #### ## Logging and dropping / resetting connections -- Loglevel 0 is emergency, 4 is warning and 7 is debug iptables -A INUNDATED -m limit --limit 5/min -j LOG --log-prefix "WEB SECURITY -- INUNDATED : " --log-level 7 iptables -A INUNDATED -m recent --set --name FROZEN --rsource -j DROP iptables -A INUNDATED -p tcp -m tcp --dport 80 -j BANDWIDTHGATES #### MINERS #### iptables -A MINERS -m limit --limit 5/min -j LOG --log-prefix "WEB SECURITY -- MINERS : " --log-level 7 iptables -A MINERS -p tcp -m recent --set --name MINER_FREEZE --rsource -j REJECT --reject-with tcp-reset ##It is better to do tcp-reset in this case instead of DROP as with PSH-ACK the connection is already established ##iptables -A MINERS -p tcp -m recent --set --name MINER_FREEZE --rsource -j DROP iptables -A MINERS -p tcp -m tcp --dport 80 -j BANDWIDTHGATES

SECURING THE S T A C K -- WEB SECURITY ######## BANDWIDTHGATES ######## ###127.0.0.0/8 is for testing locally [other IP addresses can be given as virtual interfaces for testing within a system Eg eth0:1] ###iptables -A BANDWIDTHGATES -p tcp -s 127.0.0.0/8 --dport 80 -m quota --quota 52428 -j ACCEPT iptables -A BANDWIDTHGATES -p tcp -s 10.0.0.0/8 --dport 80 -m quota --quota 10485760 -j ACCEPT ## Set a pre-defined bandwidth quota. ## 50 mbps for 192.168.150.0/24 iptables -A BANDWIDTHGATES -p tcp -s 192.168.150.0/24 --dport 80 -m quota --quota 52428800 -j ACCEPT ## 20 mbps for 172.1.0.0/16 iptables -A BANDWIDTHGATES -p tcp -s 172.1.0.0/16 --dport 80 -m quota --quota 20971520 -j ACCEPT ## 10 mbps for 10.0.0.0/8 iptables -A BANDWIDTHGATES -p tcp -s 10.0.0.0/8 --dport 80 -m quota --quota 10485760 -j ACCEPT ## 5 mbps for the rest of the IPs iptables -A BANDWIDTHGATES -p tcp --dport 80 -m quota --quota 5242880 -j ACCEPT ## When the quota is reached, the rule doesn't match any more and is dropped by the rule below. iptables -A BANDWIDTHGATES -p tcp --dport 80 -j DROP #Notes - the --update option restarts the timer on each receiving packet, where as --rcheck restarts timer only after a fixed amount of time # String search algorithms BM (Boyers Moore) is considered 2 to 6 times faster than KMP (Knuth Morris Pratt)

SECURING THE S T A C K N E T W O R K S E C U R I T Y

Aim of Accomplishment Distributed, Secure, Ad-hoc & Automatic Converged Network System Distributed - Locally (Eg. Boston - Worcester) - or Globally (Boston - San Francisco - Tokyo - Dublin - Bangalore - Beijing) Secure - Power and security of SSH - Furthermore, use of High Performance SSH especially for long distance high latency links with dynamic internal flow control buffers. Adhoc - Peer-to-Peer -- Can be connected in Ring, 2-D Torus & 3-D Hexagonal Torus Topologies, with a network diameter of up to 18 in one direction / dimension. Multiple directions / dimensions possible. Automatic - No trunking or configuring ports anymore at layer 2 / layer 3 / VLANs. Automatically bridges traffic at Layer 2 and automatically routes traffic at Layer 3 if the destination is in a different subnet / class of IP address. Converged - Acts automatically as a Switch, Router, VPN, NAC appliance, Load Balancer and more Network System -- End-to-end all-encompassing.

Aim of Accomplishment What I will not be addressing today (perhaps in another talk somewhere when possible) : How to make this existing setup into a NAC appliance with network key injection into traversing packets. How to also make this into a high performance distributed ad-hoc load balancer with auto gateway load balancing. How to logically combine different VLANS by bridging and routing them automatically at Layer 2 and Layer 3. How to tune / enhance the drivers, kernel, IP stack, bridge code, interrupts and affinities and related components to achieve line rate performance. How to route between heterogeneous physical interfaces like gigabit / 10 gigabit Ethernet and Infiniband / Scalable Coherent Interconnect Integration of Web Security methods defined before onto a layer 2 and layer 3 forwarding plane. Techniques in the Linux Kernel Networking stack to increase parallelism and performance: RSS: Receive Side Scaling RPS: Receive Packet Steering RFS: Receive Flow Steering Accelerated Receive Flow Steering XPS: Transmit Packet Steering

SECURING THE S T A C K -- NETWORK SECURITY Scenario 1 -- Converged Automatic Routing (layer 3) & Switching (layer2) SwitchR0 is a converged automatic switch & router which has 2 ethernet interfaces in a bridge swr0 swr0-192.168.200.200 swr0:1-192.168.168.1 /proc/sys/net/ipv4/conf/swr0/forwarding = 1 iptables -s 192.168.200.0/24 --table nat --append POSTROUTING --out-interface swr0 -j MASQUERADE iptables -s 192.168.200.0/24 --append FORWARD --in-interface swr0 -j ACCEPT There are 2 client machines client0 and client1 with one ethernet and one virtual interface interface each. client0 - eth0-192.168.200.111 eth0:1-172.172.172.173 connected to eth0 of SwitchR0 client 0 - default gw on eth0 set to 192.168.200.200 client1 - eth0-172.172.172.172 eth0:1-192.168.168.10 connected to eth1 of SwitchR0 client0 pings 192.168.168.10 [ on client1 ] succesfully -- layer 3 routing via SwitchR0 client0 pings 172.172.172.172 [on client1 ] succesfully -- layer2 switching via SwitchR0 As long as swr0 has IP addresses that are present on your network, layer 3 routing will be automatically and transparently possible across all inter-class IPs, inter private address IPs and their subnets!

SECURING THE S T A C K -- NETWORK SECURITY Scenario 2 -- Distributed Converged Automatic Routing (layer 3) & Switching (layer2) SwitchR0 is a converged automatic switch & router which has 1 ethernet interface & 1 tap interface in a bridge swr0 Bridge swr0 has eth0 and tap0 eth1 is given a separate IP address - 172.10.10.11 reachable to eth1-172.10.10.10 of SwitchR1 swr0-192.168.200.200 swr0:1-192.168.168.1 /proc/sys/net/ipv4/conf/swr0/forwarding = 1 iptables -s 192.168.200.0/24 --table nat --append POSTROUTING --out-interface swr0 -j MASQUERADE iptables -s 192.168.200.0/24 --append FORWARD --in-interface swr0 -j ACCEPT Spanning Tree Protocol turned on swr0 to prevent network loops: brctl stp swr0 on SwitchR1 is a converged automatic switch & router which has 1 ethernet interface & 1 tap interface in a bridge swr1 Bridge swr1 has eth0 and tap0 swr1-192.168.200.20 eth1 is given a separate IP address - 172.10.10.10 Spanning Tree Protocol turned on swr1 to prevent network loops: brctl stp swr1 on

SECURING THE S T A C K -- NETWORK SECURITY Scenario 2 -- Distributed Converged Automatic Routing (layer 3) & Switching (layer2) ssh tunnel from bridge swr0 of SwitchR0 [172.10.10.11] to swr1 [172.10.10.10] of SwitchR1 ssh -o Tunnel=ethernet -f -N -w 0:0 root@172.10.10.10 **Only change the following in /etc/ssh/sshd_config [as seen in Ubuntu]: PermitRootLogin yes PermitTunnel yes [ The above will create 2 tap interfaces (tap0 on SwitchR0 and SwitchR1) ] Same configuration of client machines as scenario 1 Client0 is connected to eth0 of SwitchR0 Client1 is connected to eth0 of SwitchR1 client0 pings 192.168.168.10 [ on client1 ] succesfully -- layer 3 distributed secure routing via SwitchR0 and SwitchR1 [over ssh] client0 pings 172.172.172.172 [on client1 ] succesfully -- layer2 distributed secure switching via SwitchR0 and SwitchR1 [over ssh]

SECURING THE S T A C K -- NETWORK SECURITY Scenario 3 -- Distributed Converged Automatic Routing (layer 3) & Switching (layer2) with Transparent Layer2 Auto Fail-over and Auto Fail-back Same setup and configuration as scenario 2 but now add SwitchR2 with a bridge swr2 and give it an IP eg: 192.168.200.40 Connect eth0 of client0 to eth0 of SwitchR2 Connect eth0 of client1 to eth1 of SwitchR2 You may use an unmanaged switches to connect client[x] to SwitchR[X] or do link aggregation (802.3ad / bonding) for NIC failover / balancing OR do everything with virtual machines As soon as all the SwitchRs are connected, they will send out BPDUs and select a root bridge Start a ping from client0 to client1 Unplug the cable from Client0 to SwitchR0 -- do: brctl showstp swr0 [ and all the switches] you notice the designated root change, meaning a new root bridge will be selected. You will also see message with content like topology change detected propagating topology change detected, sending tcn bpdu

SECURING THE S T A C K -- NETWORK SECURITY Scenario 3 -- Distributed Converged Automatic Routing (layer 3) & Switching (layer2) with Transparent Layer2 Auto Fail-over and Auto Fail-back Plug the cable back in SwitchR0 and do similar observations as above Then unplug the cable from client 0 to SwitchR2 and make observations --you should get seamless transparent fail-over and fail-back The fail-over / fail-back will be instantaneous or could take up to a minute depending on your link latency and tuning of STP parameters on the bridge

SECURING THE S T A C K O P E R A T I N G S Y S T E M S E C U R I T Y

SECURING THE S T A C K -- OPERATING SYSTEM SECURITY Access to compliant resources by authorized users from heterogeneous groups There are thousands of users spread across heterogeneous groups. You need to give only a few users across these groups access to compliant & secure resources POSIX ACLs can do that for you providing you understand its complexity The code examples show you steps to achieve 2 different scenarios to bifurcate groups and users: 1) Adding a group to a file but not giving the group members read, write or execute permissions, then adding required users from that group as need be. 2) Adding a group to a file but this time giving the entire group read, write or execute permission, then restricting access to specific users

SECURING THE S T A C K -- OPERATING SYSTEM SECURITY Scenario 1 getfacl facltest getfacl shows the file access control list(s) # file: facltest # owner: root # group: root user::rwsetfacl -m g:mehuls:--- facltest setfacl sets the file access control list. -m (modify) [-x to remove] g:mehuls gives group mehuls access to the file. --- means no rwx access getfacl facltest # file: facltest # owner: root # group: root user::rwgroup::r-x group:mehuls:--- members of group mehuls will not be able r,w or x mask::r-x other::r--

SECURING THE S T A C K -- OPERATING SYSTEM SECURITY Lets try it: su mehuls cat facltest cat: facltest: Permission denied echo "this is from mehuls" >> facltest bash: facltest: Permission denied exit Exit and give read and write permissions to user mehuls: setfacl -m u:mehuls:rw- facltest u denotes user su mehuls ls -l facltest -rw-rwxr--+ 1 root root 0 Apr 3 15:26 facltest + denotes acls have been applied echo "this is from mehuls" >> facltest cat facltest this is from mehuls

SECURING THE S T A C K -- OPERATING SYSTEM SECURITY Scenario 2 setfacl -m u:mehuls:--- facltest here user does not have permissions ( --- ) setfacl -m g:mehuls:rwx facltest group has rwx getfacl facltest1 # file: facltest1 # owner: root # group: root user::rw- user:mehuls:--- group::r-- group:mehuls:rwx mask::rwx other::r-- Try it out: su mehuls cat facltest;./facltest cat: posixacltest: Permission denied bash:./facltest: Permission denied

SECURING THE S T A C K THANK YOU!