Working with ESX(i) Log Files

Working with ESX(i) Log Files Working with ESX(i) log files is important when troubleshooting issues within the virtual environment. You can view and search log files in ESX(i) and in vcenter Server using a few different methods. Methods Using the vsphere client The direct console user interface (DCUI) A web browser A syslog or vma appliance An SSH connection to the host PowerCLI using the Get-Log command When using SSH, use the following commands to view and search the log files. Use more to page through the log files one page at a time Use tail to view the end of the log files Use grep to search Use pipe to link commands together Use pipe to grep to search through files Use cat to concatenate & use grep to search Use find print grep filename to search for a file Example Cat hostd.log grep search variable more vcenter log files vcenter log files are in a vpxd-xx.log format where xx is a numerical value that increases when each log file is 5MB in size. The log file numbers rotate when the vpxd service is started or when the log reaches 5MB in size. The log files are located in the c:programdatavmwarevmware virtual centerlogs

Other log files include Vpxd-alert-x.log Vpxd-profile-x.log Esx logs /var/log/vmkernel VMkernel messages /var/log/vmkwarning /var/log/vmksummary /var/log/vmware/hostd.log /var/log/messages service console /var/log/vmware/vpx/vpxa.log vsphere client agent /var/log/aam/vmware xxx.log- HA /var/log/vmkiscsid.log iscsi /var/log/boot-logs/sysboot.log boot log ESXi logs /var/log/messages combination of the VMkernel and vmkwarning /var/log/vmware/hostd.log host management service /var/log/vmware/vpx/vpxa.log vsphere client agent /var/log/sysboot.log boot log /var/log/vmware/aam/vmware xxx.log HA Log rotation Within ESX(i), rotation for most log files is controlled by /etc/logrotate.conf To view available options run man logrotate With both ESX and ESX(i) hostd.log rotation is controlled with /etc/vmware/hostd/config.xml Vpxa.log rotation is controlled with /etc/opt/vmware/vpxa/vpxa.cfg Should you wish to edit the rotation control files you can use Nano in ESX or vi in ESX and ESXi. I will focus on vi as I am more familiar with it and with the arrival of vsphere 5 there will no longer be ESX, and as such no native support for Nano. vi commands

a append i insert O/o open new line O is line above, o is line below r replace : search or save options / search wq write and quit x delete individual characters dd delete line $ go to the end of the line ESC break out of current mode Log bundles Log bundles can be accessed through the VMware folder on the start menu, by clicking generate vcenter server log bundle. This runs the vc-support windows scripting file located at c:program filesvmwarevirtual infrastructurevirtual centrescriptsvc-support.wsf and cscript. You can also download it through the vsphere client and by connecting to the ESX(I) server using scp with Veeam FastSCP or WinSCP. To do this you have to enable tech support mode first. An alternative way of generating log bundles is through the vm-support command run through an SSH connection to the COS or through the vma. Running vm-support will generate a tar compressed file. Procedure [root@esxhost]#/usr/bin/vm-support With ESXi it is possible to place log files on shared storage. To set this open the vsphere client connection to the host, click configuration>advanced settings>syslog select local and enter the path to the shared storage. Enter the log file location as [datastorename]/logfiles/hostname.log. vilogd

vilogd is a service that performs log collections. You can manage it with the vilogger commands. vilogger is used to enable and disable or configure the log collections with these commands. To use vilogger, first ensure that vi-fastpass is enabled using vifp list server to list out the current vi-fastpass enabled servers, if no servers are listed use vifp addserver servername and vifptarget -s servername to add again. Commands vilogger enable vilogger list vilogger update policy Control the vilogd service with etc/init.d/vmwarevilogd start stop restart vilogger has several parameters available, an example of which are numrotation number of files to collect maxfilesize specified in MB collectionperiod how often to poll, specified in seconds Example vilogger enable server servername numrotation 20 maxfilesize 10 collectionperiod 10 This command will collect the following logs from the ESXi host hostd.log messages.log vpxa.log To scroll through the log files one page at a time use the more command. Example more hostd.log

Configure vma as a Syslog Server You can configure the vma as a syslog receiver to collect log files from the ESX and ESXi server. Run the commands listed below to configure. vma #sudo service rsyslog stop #sudo nano /etc/sysconfig/rsyslog This will open nano so you can edit the following information change SYSLOGD_OPTIONS= -m 0 to SYSLOGD_OPTIONS= -r -m 0 Save and exit the file #sudo service rsyslog start #sudo iptables -I INPUT -i eth0 -p udp dport 514 -j ACCEPT #sudo nano /etc/rc.local Edit the file to add the iptables line below to the end of the rc.local file iptables -I INPUT -i eth0 -p udp dport 514 -j ACCEPT ESX To configure ESX to use vma as a syslog server add the IP address of the vma to the /etc/syslog.conf file. #vi /etc/syslog.conf Add the following lines to the bottom of the file # Send all syslog traffic to vma *.* @<IP_Address_Of_vMA> Open the firewall with #/usr/sbin/esxcfg-firewall -o 514,udp,out,syslog Finally restart the syslog service with #sbin/services syslog restart ESXi Use the vsphere client by going to configuration>advanced>syslog settings and enter into the

syslog.remote.hostname section the name of the vma. Alternatively assuming vi-faspass is enabled run #vifptarget -s [ESXihost] #vicfg-syslog -s [vma] #vifptarget -c