EAGER Rack Configuration. Overview

Transcription

1 EAGER Rack Configuration Overview Introduction This document briefly gives an overview of the second demo rack for the EAGER system. I shall describe each piece of the rack, its purpose and an overview of its operation. This project makes extensive use of VLANs, and many of the problems encountered involved the configuration of the various VLANs. Note that this document is somewhat inconsistent on the number of services available. Service 104 is currently not implemented, but some instructions about implementation details for it have been included. Server The main server hosts the virtual machines that provide the individual services. This machine contains 6 network ports, 2 on the processor board, and an additional 4 on a separate card. It contains a 1 TB hard drive, and a 256GB SSD drive. It has 4GB processor memory. Base Operating System The base operating system used on te server is Ubuntu Linux (64 bit). There is a singlue user account, called user. The software used to create the VLANS is the linux kernel vlan package. The original attempt was to use OpenVswitch, because it is compatable with OpenFlow switches, however configuring it to work properly required more extensive research than was available at the time, so the switch to vlan was made. To configure the ports for the 4 services, the following start_vlan code is used #!/bin/bash ifconfig eth up ifconfig eth up vconfig add eth2 100 ifconfig eth2.100 up ifconfig eth up vconfig add eth3 101

2 ifconfig eth3.101 up ifconfig eth up vconfig add eth4 102 ifconfig eth4.102 up ifconfig eth up vconfig add eth5 103 ifconfig eth5.103 up ifconfig eth up vconfig add eth6 104 ifconfig eth6.104 up dhclient eth1 v This code it automaticazlly executed at system startup by adding a reference to it in /etc/rc.local. The server machine only provides a platform to run virtual machines for each service, it does not provide any applications in itself. The software used on this machine for the virtual machines is VirtualBox, installed from the Ubuntu repositories. It runs under the user account. No special changes were required to this package. The images for the 4 services were copied off the Ammon machine. Services The service virtual machines running on the main server are isolated by the VLANs that they are tied to. Each services runs under a separate virtual machine, and boot up with a static IP address of Since they are all on different VLANs, the duplication of this IP address is isolated from each other. Most of the services provide a LTSP (Linux Terminal Server Project) image for the virtual machines on the Home systems to boot from. Service 100 does not, because it is a special controller service that does not directly interact with the home system displays. The original images provided by Chris were incomplete. Apparently he was attempting to rebuild them from scratch, but did not get much beyond installing Ubuntu. Instead of attempting to finish building each service, we just copies the service images from the City of Ammon cart. Service 100 Service 100 is a control service. It is used to control which service is being displayed on the Home systems. This one is special, in that it does not have an actual service displayed on the Home systems. It connects to two network interfaces, the first id eth2.100, which connects it to the

3 Switch VLAN-100 network. The other is NAT which allows it to reach the regular internet. The NAT interface is not used by this service, but is useful for maintenance purposes. The Service manager console allows an operator to communicate which service should be displayed on the Home systems. The code that accomplishes this resides in the /var/www directory for the operator interface, and the /home/user/ directory fo the code that communicates with the Home systems. To access this software, point the manager system plugged into the Service0 port to the URL Service 101 Service 101 provides weather information. This service has two network interfaces, one attached to VLAN-101 and the other attached to NAT. The NAT is not used by the software, but is useful for maintenance purposes. The home systems boot LTSP from this service, then start the captive browser EAGERBrowser, then displays the /var/www/ pages. Service 102 Service 102 provides a 911 call interface. This service has two network interfaces, one attached to VLAN-102 and the other attached to NAT. The NAT is not used by the software, but is useful for maintenance purposes. The home systems boot LTSP from this service, then starts a special version of linphone. Service 103 Service 103 provides emergency alerts, such as Amber Alerts. This service has two network interfaces, one attached to VLAN-103 and the other attached to NAT. The NAT is not used by the software, but is useful for maintenance purposes. The home systems boot LTSP from this service, then start the captive browser EAGERBrowser, then displays the /var/www/ pages. The switch provides the network intterface between the components. Most of the components interact in such a way that any switch that can handle VLAN traffic will work, except for the management systems, which need a switch capable of VLAN tagging the traffic coming from those ports. The goal for the second rack is to use an OpenFlow switch, but with so many parts of the system broken, we borrowed a Brocade switch configured just like the City of Ammon's cart. Once we get the system working using the brocade switch, we will switch back to the OpenFlow switch.

4 Brocade The brocade switch is configured with ports 1-4 for the virtual server, 5-8 for the management consoles, and 9-12 for the home systems. This one was borrowed from Ammon and was pres-configured just like the one on their cart. The server ports (1-4) are configured to pass traffic normally. Port 5 tags traffic to VLAN-100. Port 6 takes untagged traffic to VLAN-101. Port 7 tags untagged traffic to VLAN-102. Port 8 tags untagged traffic to VLAN-103. Ports 9-12 tag untagged traffic to VLAN-100. HP OpenFlow The HP OpenFloow switch is a software configured switch that allows flexibility in controlling network traffic. FloodLight Controller The FloodLight controller is a separate, small Ubuntu system running the FloodLight OpenFlow software. In the floodlight configuration file /etc/floodlight/floodlightdefault.properties, remove the line that ends with forwarding. This line causes the switch to act like a (slow) forwarding switch. Leaving this line in will actually work, but does not allow for much traffic shaping. In order to define our own (faster) static flows, the files ofboot.4.py, a python script, should be used. To get a simple look at the operation of FloodLight, poit a browser on that system to and a console display will be shown. Click on the switch ID, and it will display the ports and flows operating on the switch. The flows will display with insufficient information about the flows, but it will give you packet counts to see if the flows are doing anything. To clear out the current floodlight flows, execute the following code on the controller. curl Moodify the switch ID to match the actual switch used. Examples given in numerous tutorials sow this command with a space before the /clear, but itt will not work with the space there. A startup file called ofboot.4.py, which is a python script based on the floodlight documentation, is used to create the necessary flows to control traffic on the switch. This creates flows based on VLAN IDs, so modifying them to prioritize services should be easy. # ofboot.4.py # # Openflow (floodlight) configuration. Version 2. # # Flows arer based on vlan id's. # It uses hybrid technology. # # It is fairly fast. # import httplib import json class StaticFlowPusher(object):

5 def init (self, server): self.server = server def get(self, data): ret = self.rest_call({, 'GET') return json.loads(ret[2]) def set(self, data): ret = self.rest_call(data, 'POST') return ret[0] == 200 def remove(self, objtype, data): ret = self.rest_call(data, 'DELETE') return ret[0] == 200 def rest_call(self, data, action): path = '/wm/staticflowentrypusher/json' headers = { 'Content-type': 'application/json', 'Accept': 'application/json', body = json.dumps(data) conn = httplib.httpconnection(self.server, 8080) conn.request(action, path, body, headers) response = conn.getresponse() ret = (response.status, response.reason, response.read()) print ret conn.close() return ret pusher = StaticFlowPusher(' ') "name":"vlan-100", "vlan-id":"100", "name":"vlan-101", "vlan-id":"101", "name":"vlan-102", "vlan-id":"102", "name":"vlan-103", "vlan-id":"103",

6 "name":"vlan-104", "vlan-id":"104", "name":"vlan-99", "vlan-id":"99", HP OpenFlow Switch The HP openflow switch is a 24 port hybrid switch capable of using openflow to control the network traffic between ports. HP Supplies a manual on the internet called HP OpenFlow Switches which helps with understanding how to configure the switch. (HP Part Number: ) It helps, but many things are missing or confusing. Floodlight is currently unable to handle everything necessary for tagging specific ports to specific VLANs, therefore, the switch is programmed in a hybrid mode where the switch software controls the VLAN tagging, and the OpenFlow provides the routing. The port used for the OpenFlow controller cannot take part in OpennFlow. Ports 23 and 24 have been assiged for the controller. One port could be sufficient, but the second port allows for a secondary (backup) controller in case there is a problem with the first one. OpenFlow must be enabled on the switch, and it must know about the OpenFlow controller. This is covered in the HP openflow Switches manual, previously referenced. OpenFlow must be enabled in the switchs Port/Trunk Settings menu, for all the ports to be used for OpenFlow (ports 1-22). Leave ports disabled fr the FloodLight controller. VLAN-1 should already exist as DEFAULT_VLAN, but additional VLANs 99, 100, 101, 102, 103, and 104 must be created under the VLAN Names menu. VLANs are used for the services, and VLAN-99 is used by the OpenFlow to carry untagged traffic (normal internet traffic). In the HP OpenFlow switch, all traffic must be attached to a VLAN, so VLAN-99 is used to carry traffic that should not be VLAN tagged. Inside the switch, this

7 VLAN is tagged/untagged internally, so that when the traffic leaves the switch no VLAN tages are seen. In the VLAN Port Assignment menu, set all the ports on VLAN 1 (DEFAULT_VLAN) to no, but set port 23 nd 24 to untagged. For all other VLANs, port should be set to No. Set VLAN-99 on all ports (except 23-24) to untagged. Set VLAN-101 through VLAN-103 on all ports except to tagged. VLAN-1 (DEFAULT_VLAN) is special. This is the VLAN that handles the OpenFlow/FloodLight controller. Allocating the FloodLight controller to another VLAN as tagged or untagged will cause it to be unable to communicate with the switch. Even the telnet (putty) interface will be unavailable, requiring the serial cable interface to fix it. The VLAN-99 (untagged) allows normal traffic to flow through the OpenFlow switch. Everything in OpenFlow must go through a VLAN, and this assignes such traffic to VLAN-99. If you try to send traffic as VLAN-99 with this configuration, the switch will strup off the VLAN tag and pass it as regular traffic. With all ports that want to access the internet set as untagged on this VLAN, the traffic will appear internal to the switch as belonging to VLAN-99, bou outside of the switch the tags will be removed and it will look lik normal traffic. VLANs are allocated for the EAGER services. Here is a chart of how the types of ports are allocated. Some assignments can be set in multiple ways, so all are shown with the preferred (more restrictive) shown first. Server0-4 are 4 ethernet ports attached to the main server's virtual machines. SM1-4 are the service manager ports. Generic is for unallocated ports, or ports used for normal network access. OF Ctl is the OpenFlow controller. NAT is for a network interface box (hub) used to create a NAT firewall interface between the rack and the internet, and to provide local DHCP services for normal internet traffic. VLAN Port Assignments DEFAULT Untagged Service100 Service101 Service102 Service103 Service Server0 No No/untagged tagged No/tagged No/Tagged No/Tagged No/tagged Server1 no No/Untagged No/tagged tagged No/tagged No/tagged No/tagged Server2 no No/Untagged No/tagged No/tagged tagged No/tagged No/tagged Server3 No No/Untagged No/tagged No/tagged No/tagged tagged No/tagged Server4 No No/Untagged No/tagged No/tagged No/tagged No/tagged tagged Home No No Untagged Tagged Tagged Tagged tagged Manager No No Untagged No No No No SM1 No No No Untagged No No No SM2 No No No No Untagged No No SM3 No No No No No Untagged No

8 VLAN Port Assignments SM4 No No No No No No Untagged Generic No Untagged Tagged Tagged Tagged Tagged Tagged NAT No Untagged Tagged Tagged Tagged Tagged Tagged OF Ctl Untagged No No No No No No Here is how the ports are defined on the demo system Switch Configuration - VLAN - VLAN Port Assignment Port DEFAULT_VLAN untagged Service 0 Service 1 Service > 1 No Untagged Tagged No No 2 No No Untagged Tagged Tagged 3 No Untagged No Tagged No 4 No No Untagged Tagged Tagged 5 No Untagged No No Tagged 6 No No Untagged Tagged Tagged 7 No Untagged No No No 8 No No Untagged Tagged Tagged 9 No Untagged No No No 10 No Untagged Tagged Tagged Tagged 11 No Untagged Tagged Tagged Tagged 12 No Untagged Tagged Tagged Tagged 13 No Untagged Tagged Tagged Tagged 14 No No Untagged No No 15 No Untagged Tagged Tagged Tagged 16 No No No Untagged No 17 No Untagged Tagged Tagged Tagged 18 No No No No Untagged 19 No Untagged Tagged Tagged Tagged 20 No No No No No 21 No Untagged Tagged Tagged Tagged 22 No No No No No 23 Untagged No No No No 24 Untagged No No No No Home Systems (4 copies) These systems are the display units for the alerts. These systems attach to VLANs VLAN 100 is handled in the switch, which converts normal traffic to VLAN-100 but passes the other VLAN traffic as-is. VLAN are handled by the virtual machines themselves.

9 First install Ubuntu on then, then run the script Chris has provided to install additional software. The images Chris supplied is missing the file /home/user/eagerhypervisorclient/config.ini, so copy this file onto each home system, or they will not switch services like they should. [client_client] server_address= port=4200 program= /home/user/switch_to_vm.sh When a service is selected to be brought up front, any VirtualBox service currently up front will be minimized, and the selected service will be brought to the front. If it has not already been started, it will do a PXE boot from the service on the main server. This involves bootinh a LTSP image of Ubuntu, and running EAGERBrowser inside that image. The browser will load to display the service. Network Manager The Network manager controls which service is being displayed on the Home systems. It will switch all of the systems that it knows about to the desired service. There is no way to switch an individual machine. This machine access VLAN-100 only. The VLAN tagging is handled by the switch. Service Manager (1-3) There should be 3 service manager systems, one for each service, but we are simulating them by switching one machine into each service port. These machines attach to the VLAN defined by the service ( ), which are handled in the switch. Chris's Install is missing the file /hoe/user/config.ini [admin_client] server_address= port=4201 admin_url= client_url= allow_plugins=true [client_client] server_address= port=4200 fullscreen=false allow_plugins=true cookies="

10 [server] NAT Box admin_port=4201 client_port=4200 start_url= A home switch is used to provide DHCP services, and to interface the rack to the internet when needed. The base address of the NAT box sould be , and it sould provide DHCP addresses within that range. For normal usage, the rack does not need internet access. Service Operation Internals This is a brief overview about how the entire system operates. 1. When a Home system is started up, it sends a signal to Service-100 (using VLAN-100) to log its existence to the EAGERHypervisorServer. This keeps an internal list of Home Systems for later use. Thus, Server-100 must be started before the Home systems, and if the supervisor is restarted, then the home systems must also be restarted. 2. One of the service managers will use the web interface for that service (on a service specific VLAN) to edit the message, and post it. This stores the message in that services Virtual machine,, and makes the message available for the next time the Home browser is refreshed. 3. The network manager will select the service that will be displayed. On the service, this causes a 4201 signal (on VLAN-100), which makes the EAGERHypervisorServer image to send a signal 4200 to each Home system it knows about. This causes the home system to execute the script ****, which switches the service being displayed. 4. The Home system received the 4200 signal (on VLAN-100), causing the script **** to execute, which switches to the virtual machine that represents that service. If it is already booted, it just refreshes the browser screen. 5. If the Home Service hasn't booted yet, then it uses PXE to boot from the Service Virtual Machine (on service specific VLAN), downloading a LTSP image, through downloading EAGERBrowser. Once the browser starts, it loads the specific web site from the Service (on service specific VLAN). Usuually Booting The startup has not been completely automated yet, so his startup procedure may vary as that happens. 1. Power up the switch, and the FloodLight controller. Press button 2 on the KVM, push the power button on the small box behind the screen. Which is the FloodLight controller On the floodlight controller, start the flows usinng the command python ofboot.4.py 2. Start the server. Press button 1 on the KVM, then the power button on the server. Make sure the VLANs have started using ifconfig. If tey have not, use start_vlan to start them. 3. Start the services. On thr server, start Virtual box bye typing virtualbox &. then select each of the

11 services, and press the start button. Give them some time to boot. 4. Start the service system()s. Power them up. Start the monitering program./adminclientlauncher.sh. 5. Start the management systems. Start the system browser, and go to the URL 6. Start the 4 home systems. The order doesn't matter. They need to be started after Service-100 has completely booted.