QUICK START LINUX AGENT INSTALLATION, CONFIGURATION AND TROUBLESHOOTING GUIDELINES

Transcription

1 QUICK START LINUX AGENT INSTALLATION, CONFIGURATION AND TROUBLESHOOTING GUIDELINES Introduction: How the Client Agent works (Push Technology) The Common Agent Installation: Installing the agent from CD Installing the agent manually Configuration: Verifying and adding the agent node Configuring the agent port thru a firewall and changing port assignments Configuring the agent to autoload at bootup Verifying and licensing the agent Verifying the version and status of the agent Login/Authentication General Troubleshooting: Where the logs are located Setup environment variables Verifying the proper daemons are running with the ps command Using the netstat command to determine if the agent is hung Using the nslookup command to verify the correct server name and ip address How to put the agent in Debug Performance Connectivity Error Codes 1

2 INTRODUCTION: How the Agent works (Push Technology) All client agents use push technology, which automates the backup and restore process. The client agent contains separate internal client engines that help reduce the resource-intensive backup processes. With this feature, the client agent filters and packages its archive data for reception by the server. This data preparation and transmission method provides real-time directory browsing, offloading system resources from the backup server, improves data transfer through use of packet technology, provides network security, and monitors backup and restore jobs. The Push technology minimizes the transfer of data to the BrightStor ARCserve Backup host, because the Client Agent sends only the data that is needed for backup, thereby reducing the load on your communications network and accelerating the backup process. The host server can operate more efficiently since it no longer has to filter the data for the backup job. Basically, the Agent operates on the basis of a per job request. In this mode, the host server sends an entire list of files to the remote client at one time. The Push Agent then enables the remote client to take an active role in the process by pushing all the requested files to the host server (instead of having them pulled one at a time by the host server). The client agent browses its targeted directories, prepares the data, and transmits the data across the packet network. The backup server then prepares the data for storage on the designated backup devices. These simultaneous processes between the client workstation and the backup server create an efficient, automated backup environment. In this way, use of the Push Agent results in considerable savings in both network traffic and time, because it eliminates the series of request/acknowledge packets that otherwise have to be transmitted for each file. The Push Agent also reduces the host server s processing load by taking over the file processing task. The Common Agent The Common Agent (caagentd binary) is a standard component for all Linux client agents that are installed automatically during the first installation of any Linux client agent. The Common Agent resides in the /opt/ca/babcmagt directory. It keeps track of the client/database agents that are installed on the system in a configuration file named agent.cfg, which also resides in the /opt/ca/babcmagt directory. During the installation of a new agent, the agent.cfg file is updated with the new agent's information. You will seldom need to modify this configuration file. Manual modification of this file is required only to enable some debugging messages or to change the default TCP/IP port on which the Common Agent runs. To initiate a client agent session, the BrightStor ARCserve Backup server requests a connection for a client agent to use a specific backup component (such as BROWSER, BACKUP, or RESTORE). When it receives the request, the Common Agent accepts the connection and verifies the user s credentials for the system. Upon user validation, the Common Agent checks the agent.cfg file for an entry corresponding to that particular client agent and the specified component. Only after it has validated both the client agent and the requested component does the Common Agent activate the client agent and the component. The Common Agent then returns to a state of waiting for additional requests. 2

3 INSTALLATION: Installing the Agent from CD To install the Uagent/Common agent from CD, use the following steps: 1. On the server that you want to install the agent on, log in as root. 2. If the BrightStor ARCserve Backup packages are on CD, mount the CD on your local system. 3. To start the installation, go to the directory where you have mounted the CD, for example: # cd /mnt/cdrom 4. Go to the subdirectory for the platform you want to install the File System Agent, for example: # cd Linux 5. Run the following command: or #./install # path/install where path is the location of the install script. Note: If you have a previous version of the File System Agent installed, BrightStor ARCserve Backup detects this when you enter this command and automatically upgrades for you. Installing the Agent Manually To install the Uagent/Common agent manually, use the following steps: 1. On the server that you want to install the agent on, log in as root. 2. Create a temporary folder on the local system. # mkdir /uagent_install 3. FTP or copy the tar file to the temporary folder on your local system. 4. Untar the tar file with the following command. # tar -xvf uagentxxxx.tar 5. To start the installation, go to the directory where you transferred the tar file to, for example: # cd /uagent_install 6. Run the following commands in the following order: # rpm i ca-lic-xxxxxx.rpm # rpm i babcmagt.rpm # rpm i babagtux.rpm Note: The Common Agent must always be installed first before you install the Uagent or any special agent. 7. caagent list (to verify that the agent is enabled and running) Note: In order for a backup to be launched successfully, the needed agent must be enabled, and the common agent must be running. If the caagent list command shows the agents as enabled, but caagentd is not running you will need to launch the common agent with the caagent start command. 3

4 CONFIGURATION: Verifying and adding the agent node After the installation is complete verify that the agents are enabled and running. 1. On the server that you installed the agent on, enter # caagent list all Object_type Name Version Status 0 BABagntux ENABLED 2. If the BABagntux is not enabled then do the following. # cd path/babuagent #./uagent_setup 3. Once the agent is configured and is enabled and running, the node will have to be added to the client database on the ARCserve server. A. Start the GUI manager B. Login with caroot and select the Database Manager icon. C. The following screen will display. Click the Blue Add Agent icon, circled in yellow. 4

5 D. Select the Client tab and fill in the blank fields and click ok 4. You should now see the server/agent node in the Backup Manager screen. Configuring the Agent Port thru a Firewall MINIMUM PORTS REQUIRED FOR A FIREWALL BEB/BAB requires ports 6050 thru 6090, 80 (on server for GUI), 111 (RPC services) and to be accessible in order to backup thru a firewall. TO CHANGE PORT ASSIGNMENT FOR THE UNIX CLIENTS The default TCP port and UDP port are both The TCP port is used for communication and data transfer between the cprocess and the client agent. The Backup Manager user interface uses the UDP port to browse hosts. If you want to configure either the TCP port, UDP port, or both, the configuration files (below) must be modified on both the ARCserve server (port.cfg) and the agent side (agent.cfg). The values must match in order for communication between the server and the agent to succeed. By default, the Common Agent uses another UDP port, 0xA234 (41524), to receive BrightStor ARCserve Backup requests for the Auto Discovery of Linux client agents. This port is not configurable. An example of the port.cfg or CAportconfig.cfg file is shown below: #Hostname IP address (optional) TCP port UDP port #myhost xxx.xxx.xxx.xxx mymachine An example of the.agent.cfg file is shown below for ARCserve 10.x and above [36] #[BABcmagt] (NOTE: the # must stay for this line) NAME BABcmagt HOME /opt/ca/babcmagt TCP_PORT 6051 UDP_PORT

6 Configuring the agent to Autoload at Bootup The default selection for the ARCserve agent during the install was to allow the agent to be started by the system s runlevel control. If you manually installed the agent or chose not to allow this during the install you can easily add this functionality. Simply execute the /opt/ca/babcmagt/caagent_autostrtstop script. This will create a BAB_agent script, usually in the /etc/init.d folder Verifying and licensing the agent The license is installed with the ca-lic-xxxxxx.rpm package. You can also install the license manually using the CALicense script in the ca_lic folder. # cd /opt/ca/ca_lic #./CALicense <29 Character License Key> If you get 'invalid license' error. Verify that there is a SharedComponents and ca_lic directory under the /opt/ca folder. If not, untar the lic98.tar from cd being used for the client agent install. Change to the directory called lic98_install and run./install. This will create the SharedComponents and ca_lic directories. Rerun uagentsetup Verifying the Release Build of the agent Use the following console commands to verify the version and build of the uagent installed. # uagent -u CA BrightStor ARCserve Backup Agent for UNIX/Linux r11.5 (Build 2417)(Running on Linux el5 #1 SMP Fri Jan 26 14:15:14 EST 2007 x86_64)a Usage: uagent {status start stop} Login/Authentication Alternate users other than the root user can be used to login to the client agent and perform the backups. But keep in mind that we highly recommend that the root user be used. This insures that the user has all the permissions to access and backup the entire file system successfully. Where the Logs are Located All log messages relating to the Uagent are located in the /opt/ca/babuagent/logs directory. cabr.log uag.log uag.ipc All log messages relating to the Common Agent are located in the /opt/ca/babcmagt/logs directory. caagentd.log 6

7 GENERAL TROUBLESHOOTING: Setup Environment Variables A sample structure of the agent.cfg file with a client agent installed is shown below: File Contents Field Description [0] Object type: pre-defined number of a specific agent #[BABagntux] Section Name NAME BABagntux Agent s name VERSION nn.nn.nn Agent s version number HOME /opt/ca/babuagent Home directory for agent ENV CA_ENV_DEBUG_LEVEL=4:$CA_ENV_DEBUG_LEVEL Environment variables passed to agent ENV LD_LIBRARY_PATH=/opt/CA/CAlib:/opt/CA/BABcmagt: $LD_LIBRARY_PATH ENV SHLIB_PATH=/opt/CA/CAlib:/opt/CA/BABcmagt: $SHLIB_PATH ENV LIBPATH=/opt/CA/CAlib:/opt/CA/BABcmagt:$LIBPATH BROWSER cabr Browser module for agent AGENT uagentd Backup module for agent daemon MERGE umrgd Merge daemon VERIFY umrgd Scan daemon [36] DISABLED #[BABcmagt] #NAME BABcmagt #HOME /opt/ca/babcmagt #TCP_PORT 6051 #UDP_PORT 6051 How to put the Agent in Debug Use your preferred text editor to edit the agent.cfg file in the common agent directory (/opt/ca/babcmagt) and remove the # from the line that says debug level 4 like the example below. The complete line is: #ENV CA_ENV_DEBUG_LEVEL=4 Stop and start agent before doing backup. The debug output will be written to the caagentd.log file in the BABcmagt/logs directory. Verifying the Version and Status of the Agent Use the following console commands to verify the status of all the agents installed. # caagent list all Object_type Name Version Status 0 LinuxAgent ENABLED # caagent disable all (will disable all the active agents on the list) # caagent enable all (will enable all the disabled agents on the list) # caagent enable <object> (will enable a specific object/agent on the list) 7

8 Verifying the proper daemons are running With no backups active- one caagentd daemon running # ps -ef grep agent root ? 00:00:00 /usr/bin/ssh-agent /. root :22? 00:00:00 /opt/ca/babcmagt/caagentd root :52 pts/2 00:00:00 grep agent With no backups but the manager browsing two caagent daemons running and one cabr running- note the parentchild relationship with the starting caagentd process # ps -ef grep agent root ? 00:00:00 /usr/bin/ssh-agent /.. root :22? 00:00:00 /opt/ca/babcmagt/caagentd root :06? 00:00:00 /opt/ca/babcmagt/caagentd root :06 pts/2 00:00:00 grep agent # ps -ef grep cabr root :06? 00:00:00 cabr root :07 pts/2 00:00 grep cabr With one backup running while there are 2 or more uagentd processs, note that the parent-child relationship can be followed back to the caagentd process that launched it # ps -ef grep agent root ? 00:00:00 /usr/bin/ssh-agent / root :22? 00:00:00 /opt/ca/babcmagt/caagentd root :10? 00:00:11 uagentd root :10? 00:00:00 uagentd root :11 pts/2 00:00:00 grep agent Connectivity Using the nslookup command to verify the correct server name and IP address On both the Host and Client Agent server, run the command: Ex: by server name # nslookup <servername> Server: test01.com Address: Name: <servername>.com Address: Ex: by IP address # nslookup Server: test01.com Address: Name: <servername>.com Address:

9 Using the netstat command to determine if the agent is hung On the Client Agent server, run the command: # netstat -an grep 6051 tcp : :* LISTEN udp : :* Note: The agent must be loaded (even if hung) to see anything with netstat. If the result you get is a 'closed wait' or 'wait' status for port 6051, then the agent process is hung. Netstat can also be done on the windows side to verify if it is communicating with the client agent. Trouble shooting windows connections Get the following information? 1) From the Windows server: hosts file ipconfig /all tracert <linux machine by name> tracert <linux machine by IP address> nslookup <linux machine by name> nslookup <linux machine by IP> 2) From the linux machine: /etc/hosts /etc/nsswitch.conf ifconfig -a hostname traceroute <win machine by name> traceroute < win machine by IP address> nslookup <win machine by name> nslookup <win machine by IP address> 3) On the linux machine: caagent disable all Make sure that port 6051 is clear and that no caagent process is running. cd /opt/ca/babcmagt mv logs logs_<date> cd to the uagent home directory, put the uagent in single user mode (-S in uag.cfg) caagent enable all 4) Try to login again. If no information is written to the common agent logs about the login attempt, our connection is being denied. Send in or check the iptables config file normally in the /etc/sysconfig folder Also, get a netstat -an and o output from both sides. 9

10 Performance Typically people experiencing throughput problems have run benchmark tests on the same set of data between a command-line backup utility or have noticed what they believe to be slow throughput in their ARCserve backups. The following steps describe a troubleshooting process for throughput issues. 1. On ARCserve: Check the activity log on the backup server ($BAB_HOME/logs/BrightStor.log). For the relevant backups, how many files were backed up, what was the total file size of the backup, and what was the average throughput? If possible, simply include the relevant sections directly from the log file. 2. In order to increase the performance, we have to consider possible bottlenecks. This include physical connection connections to the backup devices, backup device speed, network connections, and disk I/O speed. One possible test is to use the tar system utility for a rough benchmark. How long does it take for tar to archive the identical files to tape? This can be determined by prefacing the tar command with the time command. (e.g. time tar -cvf /dev/rmt/0h /tmp/test_dir) 4. The Unix agents provide several command-line arguments that may increase the throughput of data to the ARCserve engine. Keep in mind that increasing performance will decrease available network bandwidth, and that overall throughput to tape is dependent on a number of factors such as (but not limited to) processor power; number of, layout, and type of SCSI devices and interfaces; average size of files being transferred; and network usage and capacity. Command-line options are entered after the start command, as shown below: # uagent start -b c 1 -m b [ ] Controls the size of the disk I/O buffer, in bytes. The default is Increase this value to improve performance. -s [ ] Specifies the size of the socket buffer. Enter a value from 4096 to c [0-1000] Controls the sleep waiting time-out when shared memory is empty or full. The default is 0, which allows the agent to dynamically control the sleep time-out. Setting this value to 1 will try to devote all available CPU processing to the uagent, and setting it to 1000 will make it more resource-friendly. A value of 1 is suggested when more resources are needed. -m [ ] Controls the number of disk I/O buffers. The default for SCO, UnixWare, and Solaris x86 is 100, and is 256 for all other platforms. Increase this value to improve performance. UAG.CFG default parameters -b bufsize Defines the disk I/O buffer size in bytes. Enter a value from 2048 to bytes. Default: c n Defines the sleep time in milliseconds (ms), while waiting for a shared memory slot to become completely full or empty. Enter a value from 0 to 1000 ms. Default: 0 -m maxbuf Defines the number of buffers allocated for I/O. Specify from 2 to 1024 buffers. Default: 128 -s bufsize Specifies the size of the socket buffer. Enter a value from 4096 to Default:

11 Error Codes E8511 Failed to connect to the client agent. Verify correct user name and password Disable the personal firewall as a test Verify ca_auth equivalency for the owner/user Verify healthy network - NIC cards and cable E8522 Failed to receive data from client ec=-1 This is a connection or network issue and can be exactly that. If the ARCserve server is on the Windows platform, this type of error can be confirmed by looking in the event viewer and seeing "broken-pipe" errors. In this case you will need to verify and correct basic network connectivity, packet corruption and router status Ping the by ip address and server name both ways Verify healthy network - NIC cards and cable Download CA doc ID: TEC Check to see if this document is still available and current location. E8524 Received unrecognized command from the client agent. Verify correct builds of the agent and base Base product should be of higher version/build than that of the agent Disable the personal firewall as a test E8531 Request denied by the client agent Verify correct user name and password Verify correct user permissions E8535 Failed to receive data from client agent Verify correct option usage in the uag.cfg file Comment out the line #ENV LD_ASSUME_KERNEL= from /opt/ca/babcmagt/agent.cfg file and restart the client agent services Ping the by ip address and server name both ways Verify healthy network - NIC cards and cable E9004 Failed to connect to agent <server name>, Is the agent loaded and the TCP port correct? Fairly self-explanatory, this usually means that the uagent is unreachable on the client machine because it has either not been installed, started, or there has been a communications error. There are rare instances in which bad IP address information is passed to the job queue, which will return an abnormal or otherwise incorrect IP address. Enable portmapper and the udp port Disable the firewall as a test Ping the by ip address and server name both ways Verify healthy network - NIC cards and cable 11

12 E9011 Failed to receive packet Address already in use Ping the by ip address and server name both ways Check if the following settings are to low These registry settings are in the Win2k, NT 4, and Win 95/98 Client Agent registry under: HKLM\SOFTWARE\Computer Associates\Brightstor Arcserve Backup\ClientAgent\Parameters: Change the values of ReconectionTimeOut, ReceiveTimeOut and SendTimeOut to the desired number of seconds Verify healthy network - NIC cards and cable E9014 Failed to receive header packet [xxxxxxx: No such file or directory] Ping the by ip address and server name both ways Verify healthy network - NIC cards and cable Lost connection to (machine name) (IPaddress). Do you want to connect? Comment out the line #ENV LD_ASSUME_KERNEL= from /opt/ca/babcmagt/agent.cfg file Getting "invalid command" after issuing "uagent start" Verify that the environment variables are set cd to the BABuagent directory and run uagentsetup Getting the error "Failed to login the agent" even after putting the correct root password. Comment out the line #ENV LD_ASSUME_KERNEL= from /opt/ca/babcmagt/agent.cfg file Getting "failed to connect to merge/scan daemon" at the end of backup. Verify that the environment variables are set cd to the BABuagent directory and run uagentsetup Uagent is not running, scan/merge operation needs uagent up and running. Getting uagent is not registered when trying to load the agent. Verify that the environment variables are set cd to the BABuagent directory and run uagentsetup and reply YES to reregister the uagent 12

13 W300014: Failed to get semaphore. Increase the following kernel parameters: set shmsys:shminfo_shmmax= set shmsys:shminfo_shmmni= set shmsys:shminfo_shmseg= Settings will vary according to the server and amount of memory available. The following parameters should also be checked and may need increasing. set semsys:seminfo_semmni= set semsys:seminfo_semmns= set semsys:seminfo_semmnu= set semsys:seminfo_semmsl= set semsys:seminfo_semopm= set semsys:seminfo_semume= set semsys:seminfo_semvmx= The minimum suggested parameters can be found in the BrightStor for UNIX INSTALLATION and RELEASE NOTES Slow Restores. cstop Follow Doc TEC verify document availability and location. If the ARCserve server is on the Windows platform: Increase the value for WSRcvdBuffSize and WSSendBuffSize in the registry. HKEY_LOCAL_MACHINE\SOFTWARE\ComputerAssociates\BrightStor ARCserve Backup\Base\Task\Remote to cstart 13