CS179i: Guide - Virtual Machine Setup and Internal Networking in Alpha Lab This guide provide guidance on critical steps on using VirtualBox to setup a virtual machine (VM) on configure external (internet providing) and internal (inter-vm) networking for VMs. Updated 4/18/2012 Virtual Machine Creation. [1] Goal: Create a new Linux virtual machine, on the local drive, with 4 GB of disk space and 256 MB of memory. Procedure: Create New VM. 1) Open a terminal and start the VirtualBox GUI by running 'VirtualBox &'. 2) Configure the host network to allow VMs to access the internet. a) Under file->preferences->network->vboxnet0 network, click on the icon that looks like a screwdriver, and enter the following information: b) IPv4 Address: 192.168.0.1 c) IPv4 Netmask: 255.255.255.0 d) Configure a new VM as follows: e) Provide your VM with a descriptive name which includes your login. We recommend the format <network identifier>_<node identifier> (i.e. green_node1, green_switch1, red_router1, etc.). f) Operating system: Linux g) Version: Red Hat h) Base Memory Size: 256 MB i) Create a new fixed size storage hard disk and make sure the Boot Hard Disk option is selected. j) The save location should be /extra/<your login>. You shouldn't create the virtual machine in your home directory, or the setup will fail you don't have 4 GB of space in your home directory! k) Once the image is created, double click the 'Network' header in its Details tab. On the resulting menu set Adapter 1 to Attached to: Host-only adapter. CentOS Linux installation Goal: Install CentOS linux from an iso (CD) image that's stored in the '/extra' directory. Procedure: Install Linux. 1) Verify that CentOS 5 CD 1 is under /extra/centos-5.7-i386-bin-1of8.iso. 2) In the VirtualBox GUI, double click on the Storage header. 3) Highlight 'Emtpy', under 'IDE controller'. 4) Look over to the right onto the storage management interface and you should see a section titled 'Attributes'. Click on the folder icon next to the 'CD/DVD' section, and add the iso mentioned in 1. Hit OK. 5) Start the virtual machine. It will boot off of the ISO image that you specified.
6) Type in 'linux text' into the boot screen. 7) Follow instructions. The critical steps are listed here in the order you will encounter them: a) Skip the media test; it's irrelevant - we are not using physical media. b) Say yes if warned to erase all data. c) For Partitioning Type, select Create custom layout. d) The size and type of partitions should be as follows: Mount Point Type Size / ext3 3500M Swap 500M e) Ignore the partition warning. f) Set up the bootloader i) Use grub ii) No password iii) Boot CentOS on /dev/sda1 iv) MBR g) Configuring the network interface for internet access. i) Yes to eth0 ii) Activate on boot iii) Enable Ipv4 support iv) Ipv4 connfiguration for eth0 v) Manual address configuration: vi) IP Address: 192.168.0.x. Where 1 < x < 255 and is unique amongst your VMs. vii) Netmask: 255.255.255.0 viii) Gateway: 192.168.0.1 This is the IP address from the first section in step 2b. ix) Primary DNS: 138.23.169.10 (the CSE DNS address). h) Hostname configuration i) Select manually. ii) Use a hostname that is unique to you and is descriptive: <your login>-<node id> (i.e. bcampana-greennode1). i) Time Zone Selection i) Uncheck System Clock Uses UTC ii) Use America/Los_Angeles j) Pick a good root password. k) Package selection will require minimal selection. Remember, we only have the first of eight CDs of the install! If you don't do this correctly, you will need to go back and fix it. i) Deselect all options then check Customize Now in software selection ii) Deselect all packages in every category l) Wait while the install completes. m) Once the install has finished, reboot the system. You may see that the system attempts to boot off the CD again. If this is the case: i) Close the window for the VM by going to Machine -> Close, selecting Power off. ii) Go into the System settings, and change the boot order so that the hard disk is first. n) Log in as root. o) Turn of SELinux by editing /etc/selinux/config. If you are not familiar with VI, you can use another text editor such as nano or emacs. Change the line in the file 'SELINUX=enforcing' to 'SELINUX=disabled'. p) Reboot. q) Update all packages by running 'yum update'. r) Create an account for yourself using 'useradd'. 'man useradd' for options if you want to do something other than the default. You will need to set up a password for the account using the 'passwd' command. s) Give your new account sudo permission. i) Install sudo with yum install sudo.
ii) Add the line <username> ALL=(ALL) ALL to the /etc/sudoers file. t) Log in with the new account to make sure it works. Inter-VM networking. Switched nodes. [4] Goal: Setup a switch connected to three nodes. Figure 1 shows the target topology. Node 1 Switch 1 Node 2 Node 3 Figure 1 - Target topology. Procedure: Setup VirtualBox for Switched Network. 1) Setup four new VMs. We will name them: a) green_node1 b) green_node2 c) green_node3 d) green_switch1 2) Setup each of the three nodes to have a second network adapter to connect with the switch. a) Open a node s Network settings and go to the adapter 2 tab. b) Enable the network adapter. c) Set Attached to: to Internal Network. This will instruct VirtualBox to automagically connect all devices with the same internal network name. d) Set the name of the internal network, Name, to something descriptive and unique to this node (i.e. linknode1, linknode2, linkredrouter1, etc.). e) Note the internal network name, you will need to re-use it when setting the switch/router on the other end of the link. Figure 2 shows the network with their VirtualBox internal network names annotated. f) Since we will be working with both IP and MAC addresses, you may find it useful to set each MAC address to a recognizable value. (i.e. for node 12 in network 5, 08:00:00:00:05:12)
Node 1 linknode1 linknode2 Switch 1 linknode3 3) Node 2 Node 3 Figure 2 - Network with VirtualBox internal network names (red boxed). 4) After each node is configured, create three network adapters (one for each node) on the switch VM. a) Open the switch s Network setting. b) Enable a new Internal Network adapter for each of the previously named internal networks (e.g. linknode1, linknode1, and linknode1). You can select the internal network names from the drop down menu. Procedure: Setup the network script for each internal network adapter. 1) Start a node VM. We will setup each node s internal network adapter to have a proper IP address. a) As shown in Figure 3, we will assume IP address in our green network will have the format: 192.168.1.xxx. b) Remember that 192.168.0.xxx/24 is used for our host connections. green network Network ip: 192.168.1.0 192.168.1.1 Netmask: 255.255.255.0 Broadcast: 192.168.1.255 192.168.1.2 192.168.1.3 Figure 3 - network example IP settings. 2) See what network adapters are currently setup in the VM by using ifconfig: type ifconfig -a. a) The -a will cause disabled adapters to also display. man ifconfig for other usages. b) You should see three adapters.
i) eth0 The VM s connection to the host. Used for internet access. ii) eth1 The VM s connection to the internal network. iii) And a loopback. 3) Navigate to the location of the network scripts: cd /etc/sysconfig/network-scripts. This folder contains scripts pertaining to network settings. To learn more about further, do an online search for something like centos network scripts or try this document. 4) Edit the script which configures this VM s adapter into the internal VirtualBox network. If you have been following this guide, it is most likely named eth1 so we will assume this. a) Open the interface configuration script for eth1 named ifcfg-eth1. b) Modify the relevant lines: BOOTPROTO=static c) Referring to Figure 3 for an example of assignments; add an IP address, broadcast, netmask, and network assignments to the script: IPADDR=<ip address of node> NETMASK=<netmask of network> NETWORK=<network ip> BROADCAST=<broadcast ip> d) Here is an example completed script: # eth1 device setting for node 1 in green network. DEVICE=eth1 BOOTPROTO=static ONBOOT=yes HWADDR=08:00:00:00:01:01 IPADDR=192.168.1.1 NETMASK=255.255.255.0 NETWORK=192.168.1.0 BROADCAST=192.168.1.255 e) Reboot your network: /etc/init.d/network restart f) Use ifconfig to verify your settings worked. If so, your vm will run these scripts whenever the network initializes (i.e. on boot). g) Repeat these steps for each node VM. Modify settings as necessary. Procedure: Setup the switch VM to be a switch using the bridge kernel module and the brctl tool. 1) Boot up your switch VM. 2) The bridge kernel module handles bridging network functions in CentOS. Remember this kernel module, we will be making modifications to it later in this course. 3) The bridge kernel module is pre-installed, but we must install the bridge utilities package in order to get the bridge configuration tool, brctl. Type: yum install bridge-utils.
4) If you run ifconfig, you will notice five network interfaces (one host connection, three internal connections, and a loopback). There will also be additional network interface script files, one for each of the interfaces. 5) Add a new configuration script for the new bridge interface, we will name it br0. Open a new file: ifcfg-br0. 6) Here is a sample bridge configuration script, make sure to modify the network settings as needed: # switch 1, bridge 0 (br0) settings. DEVICE=br0 TYPE=Bridge BOOTPROTO=static NETWORK=192.168.1.0 NETMASK=255.255.255.0 GATEWAY=192.168.1.254 ONBOOT=yes 7) Notice that the gateway IP address is that of a non-existing machine. We will configure this machine later when we deal with routing networks. 8) We will make a bridge out of the three internal interfaces: eth1, eth2, and eth3. These interfaces do not require any IPv4 settings, so we just have to set their bridge identity. a) Open an interface configuration script for one of the three internal adapters. b) Modify the script to resemble: # Switch VM, eth1 settings. DEVICE=eth1 HWADDR=08:00:27:1E:1E:BA ONBOOT=yes TYPE=Ethernet BRIDGE=br0 c) Notice the BRIDGE setting. When this script runs, it will use the brctl tool to associate this interface with our created bridge, br0. d) Repeat with remaining internal network interface scripts. 9) Reboot your network. 10) Run ifconfig to see that your Ethernet interfaces are up as well as your newly created bridge. 11) You can use brctl to view bridge settings, type: brctl show. a) You should see that bridge br0 is associated with interfaces eth1, eth2, and eth3. Great! b) man brctl for other usages and bridge manipulations. Procedure: Test your new switched network. Send pings, sniff packets. 1) From green node 1, send a ping packet to green node 2. a) From the green node 1 VM, type ping 192.168.1.2. b) You should see the responses will no loss packets. 2) Setup a packet sniffer on the switch to monitor activity on the bridge interface. a) Install tcpdump. Type yum install tcpdump. b) tcpdump outputs information of packet activity on network interfaces.
c) On the switch VM, type tcpdump -i br0 -e -nn. i) -i specifies which interface to monitor, in this case our bridge (and the associated interfaces eth1, eth2, and eth3). ii) -e causes printing of the link level header of the packet (Ethernet header in this case). iii) -nn makes for easier reading. iv) man tcpdump for other options. 3) Now send one ping packet from green node 1 to green node 2. a) From green node 1, type ping -c 1 192.168.1.2. b) You will see several logs from tcpdump within the switch VM. There should be only two types, ARP (x806) and IP(x800). The log format you see is: <time stamp> <src MAC> <dst MAC> <ethertype> <payload size> <payload> 4) On other nodes, you can also monitor packets from their (probably called) eth1 interface. Building Linux Kernel Modules. [2][3] Goal: Build your own bridge kernel module. Procedure: Get the source. 1) Open the VM of your green switch. 2) As root, install the necessary packages. a) Run yum install unifdef make gnupg gcc ncurses-devel rpm-build m4 redhat-rpm-config 3) Switch to another user. You should not compile kernels modules while root. 4) Setup the directory tree. a) Run mkdir -p ~/rpmbuild/{build,buildroot,rpms,sources,specs,srpms}. b) Also run echo '%_topdir %(echo $HOME)/rpmbuild' > ~/.rpmmacros. 5) Get the kernel source packages. a) Head over to http://vault.centos.org/5.7/os/srpms/. b) Find the link for the kernel source RPM, named kernel-2.6.18-274.el5.src.rpm. c) Run rpm -i <link to kernel source rpm> 2>&1 grep -v mockb 6) Source package is now installed. Unpack and prepare the source files. a) Change directories to find the kernel spec, cd ~/rpmbuild/specs. b) Unpack the rpm, rpmbuild -bp --target=`uname -m` kernel.spec 2> prep-err.log tee prepout.log i) Note that those are back ticks around the uname call. c) Check prep-err.log for any errors. 7) The kernel source root directory is now in: ~/rpmbuild/build/kernel-2.6.18/linux-2.6.18-274.el5.i686/ 8) Change directory to the source root. 9) Run the following commands to setup the compilation. a) make oldconfig. b) make menuconfig. You can immediately exit and save the from the menu. Our required settings are enable by default.
c) make prepare. d) make modules_prepare. Procedure: Place a debug log into the bridge module. 1) As it is common to print message to stdout, we will add two lines of code to the bridge module to demonstrate this. 2) Open net/bridge/br_input.c in a text editor. 3) Find the br_handle_frame function. 4) After sk_buff is initialized, insert the lines: be16 ethertype = eth_hdr(skb)->h_proto; printk(kern_alert Backwards hex ethertype: %x\n ); 5) An brief explanation of those two lines: a) skb is a socket buffer. It contains the data that came through the socket (headers, payload, etc.). b) eth_hdr() returns the Ethernet header from the buffer. c) h_proto is the header s ethertype value. d) printk() does logging for the kernel. e) The macro KERN_ALERT causes the log to print to stdout. 6) Read through the code to learn more about related functions. A searchable guide to the RedHat source exists here. Though this code may be different from the CentOs source you currently have. I m sure there exists a CentOs equivalent online. Procedure: Compile the bridge module. (Original documentation) 1) Build the bridge module. To build any module just locate its directoy (anywhere in your system). a) Since we are currently in the kernel source root, the path to the bridge module is net/bridge. b) Run make M=net/bridge. 2) There is now a compiled.ko in the module directoy. 3) As root, copy net/bridge/bridge.ko to /lib/modules/`uname -r`/extra. a) Don t forget those are back ticks around uname. Procedure: Load modules located in extra/ directory. 1) We d like to use modules that have been put into the /lib/modules/`uname -r`/extra directory. 2) Switch to root user. 3) First, stop services which may use our module. In this case, the network. a) /etc/init.d/network stop. 4) Unload and remove the current bridge module. a) rmmod bridge. 5) Load the module in the extra/ directory. a) depmod a. 6) You can use modprobe as root to see if your module is properly loaded at any time. a) modprobe l bridge. b) The path to your module under the extra/ directory should be displayed. 7) If you re changes are loading, just reboot and your module should be loaded then.
8) Test your module by sending packets through your switch. a) Ping from one node to any other node. b) You will see outputs in the switch VM with values 608 and 8. c) printk is dropping trailing and leading zeros. The ethertype s endianess is also backwards, that is why the ARP ether type is listed as 608 rather than 806. References 1. CS 183: System Administration lab instructions. 2. HowTos/ I Need the Kernel Source CentOS Wiki. http://wiki.centos.org/howtos/i_need_the_kernel_source 3. HowTos/ Building Kernel Modules CentOS Wiki. http://wiki.centos.org/howtos/buildingkernelmodules 4. Many, many searches on Google or your favorite search engine.