VERITAS SNMP Extension 4.0 Administrators Guide For Windows and UNIX
Disclaimer The information contained in this publication is subject to change without notice. VERITAS Software Corporation makes no warranty of any kind with regard to this manual, including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose. VERITAS Software Corporation shall not be liable for errors contained herein or for incidental or consequential damages in connection with the furnishing, performance, or use of this manual. Copyright Copyright 1999-2003 VERITAS Software Corporation. All rights reserved. VERITAS, VERITAS Software, the VERITAS logo, VERITAS SNMP Extension, and all other VERITAS product names and slogans are trademarks or registered trademarks of VERITAS Software Corporation. VERITAS, the VERITAS Logo, and SNMP Extension Reg. U.S. Pat. & Tm. Off. Other product names and/or slogans mentioned herein may be trademarks or registered trademarks of their respective companies. VERITAS Software Corporation 350 Ellis Street Mountain View, CA 94043 USA Phone 650 527 8000 Fax 650 527 2901 www.veritas.com
Contents Preface...............................................................1 Audience and Purpose of this Document....................................... 1 Conventions............................................................... 2 Typographic Conventions................................................ 2 Notes and Cautions...................................................... 2 Key Combinations....................................................... 2 Chapter 1. Introduction..................................................3 Description of VERITAS SNMP Extension..................................... 4 Architectural Overview...................................................... 4 Types of Events.......................................................... 5 Supported SNMP Compatible Frameworks.................................... 7 Supported Platforms........................................................ 7 List of Features............................................................. 8 Chapter 2. Installation..................................................11 System Requirements......................................................12 Supported Platforms.................................................... 12 Required Packages...................................................... 12 Pre-Installation Questions to Consider........................................ 13 Server Information...................................................... 13 SNMP Framework...................................................... 13 Monitoring Information................................................. 13 Where to Get VERITAS SNMP Extension Software............................. 14 iii
Installing VERITAS SNMP Extension on UNIX................................ 14 Installing VERITAS SNMP Extension on Windows............................ 15 SNMP Extension Files Available After Installation............................. 16 Support and Distribution................................................... 18 Chapter 3. Standard Frameworks Configuration........................... 19 HP OpenView Network Node Manager SNMP Event Configuration............. 20 HP ITO Integration Steps................................................... 21 Allowing New Trap Definitions.......................................... 21 IBM/Tivoli NetView or TEC Configuration Steps.............................. 22 CA UniCenter TNG Enterprise Manager Configuration........................ 23 Example.............................................................. 23 Chapter 4. Customization.............................................. 27 Required Customizations................................................... 28 Customizing snmpd.conf.............................................. 28 Customizing notify.conf............................................. 31 Optional Customizations................................................... 34 Customizing snmp.conf................................................ 34 Customizing freq.conf................................................ 35 Customizing messages.conf........................................... 38 Customizing process.conf............................................ 40 Customizing collector.conf.......................................... 42 Customizing vcs.conf................................................. 44 Customizing vxvm.conf................................................ 45 Customizing vxfs.conf................................................ 45 Customizing vsm.conf................................................. 45 Agent Usage.............................................................. 46 Appendix A. Common Events........................................... 49 Most Common Events..................................................... 49 iv Administrators Guide
Condition/Threshold Based Events.......................................... 50 Critical Process Based Events................................................ 50 Event Samples............................................................ 50 Appendix B. Supported NetBackup Error Conditions........................53 Monitored Standard NetBackup Error Conditions and Error Codes.............. 54 Condition/Threshold Based Events.......................................... 54 Critical Process Based Events................................................ 55 1-254. Name: abnormal_backup_exit_status.................................. 55 300. Name: DB_backup_failed............................................... 55 301. Name: DB_backup_is_disabled.......................................... 55 302. Name: freezing_media................................................. 56 303. Name: suspending_media.............................................. 56 304. Name: media_required_for_restore...................................... 57 305. Name: downed_drive.................................................. 57 306. Name: exceeded_cleaning_frequency..................................... 57 307. Name: exceeded_max_mounts.......................................... 58 308. Name: mount_request.................................................. 58 309. Name: no_cleaning_tape................................................ 59 310. Name: zero_cleanings_left.............................................. 59 Appendix C. Supported VxFS Error Conditions.............................61 Supported VxFS Capacity and Rules Related Error Conditions................... 62 1. VxFS file system full..................................................... 62 2. VxFS Defragmentation................................................... 62 3. VxFS Damaged.......................................................... 62 4. VxFS license............................................................ 62 5. VxFS free inodes is critical................................................62 Supported VxFS Standard Error Codes....................................... 62 Code: 001................................................................. 62 Code: 002................................................................. 63 Contents v
Code: 003, 004, 005........................................................ 63 Code: 006, 007............................................................ 64 Code: 008, 009............................................................ 64 Code: 010................................................................ 65 Code: 011................................................................ 65 Code: 012................................................................ 66 Code: 013................................................................ 66 Code: 014................................................................ 66 Code: 015................................................................ 67 Code: 016................................................................ 67 Code: 017................................................................ 67 Code: 019................................................................ 70 Code: 020................................................................ 70 Code: 021................................................................ 70 Code: 022................................................................ 71 Code: 023................................................................ 71 Code: 024................................................................ 72 Code: 025................................................................ 72 Code: 026................................................................ 72 Code: 027................................................................ 73 Code: 028................................................................ 73 Code: 029, 030............................................................ 73 Code: 031................................................................ 74 Code: 032................................................................ 74 Code: 033................................................................ 74 Code: 034................................................................ 75 Code: 035................................................................ 75 Code: 036................................................................ 75 Code: 037................................................................ 76 Code: 038................................................................ 76 vi Administrators Guide
Code: 039................................................................. 77 Code: 040................................................................. 77 Code: 041................................................................. 78 Code: 042................................................................. 78 Code: 043................................................................. 78 Code: 044................................................................. 79 Code: 045................................................................. 79 Code: 046................................................................. 79 Code: 047................................................................. 79 Code: 048, 049............................................................. 80 Code: 050................................................................. 80 Code: 051................................................................. 80 Code: 052................................................................. 81 Code: 053................................................................. 81 Code: 054................................................................. 81 Code: 056................................................................. 82 Code: 057................................................................. 82 Code: 058................................................................. 83 Code: 059................................................................. 83 Code: 060................................................................. 83 Code: 061................................................................. 84 Code: 062................................................................. 84 Code: 063................................................................. 84 Code: 064................................................................. 85 Code: 066................................................................. 85 Code: 067................................................................. 85 Code: 068................................................................. 86 Code: 069................................................................. 86 Code: 070................................................................. 86 Code: 071................................................................. 87 Contents vii
Appendix D. Supported VCS Error Conditions............................. 89 Supported VCS Critical Events.............................................. 90 Supported VCS Critical Daemon Process Error Conditions...................... 90 Supported VCS Standard Error Codes........................................ 90 Code: 1000............................................................... 90 Code: 1005............................................................... 90 Code: 1010............................................................... 91 Code: 1015............................................................... 91 Code: 1020............................................................... 91 Code: 2000............................................................... 91 Code 2001................................................................ 92 Code: 2002............................................................... 92 Code: 2003............................................................... 92 Code: 2004............................................................... 92 Code: 2005............................................................... 93 Code: 2006............................................................... 93 Code: 2007............................................................... 93 Code: 2008............................................................... 93 Code: 2009............................................................... 94 Code: 2010............................................................... 94 Code: 2011............................................................... 94 Code: 2012............................................................... 94 Code: 2013............................................................... 95 Code: 2014............................................................... 95 Code: 2015............................................................... 95 Code: 2016............................................................... 96 Code: 2017............................................................... 96 Code: 2018............................................................... 96 Code: 2019............................................................... 96 Code: 2020............................................................... 97 viii Administrators Guide
Code: 4000................................................................ 97 Code: 4001................................................................ 97 Code: 4003................................................................ 97 Code: 4004................................................................ 97 Code: 4005................................................................ 97 Code: 4006................................................................ 98 Code: 4007................................................................ 98 Code: 4008................................................................ 98 Code: 4009................................................................ 98 Code: 4010................................................................ 98 Code: 4011................................................................ 99 Code: 4012................................................................ 99 Code: 4013................................................................ 99 Code: 4014................................................................ 99 Code: 4015................................................................ 99 Code: 4016................................................................ 99 Code: 4017............................................................... 100 Code: 4018............................................................... 100 Code: 4019............................................................... 100 Code: 4020............................................................... 100 Code: 4021............................................................... 100 Code: 4022............................................................... 100 Code: 4023............................................................... 101 Supported VCS Standard Error Conditions for Windows NT/2k platform........ 102 Failed to create directory.................................................. 102 Insufficient Memory......................................................102 Could not add privileges.................................................. 102 The name of the local host is invalid......................................... 102 Failed to save the registry................................................. 102 Failed to restore the registry................................................ 102 Contents ix
The main (root) key is unknown............................................ 103 Failed to get the computer name........................................... 103 Duplicate key, or already part of notification................................. 103 Delete and Restore not attempted as backup file cannot be created.............. 103 Open registry keys........................................................ 103 Agent module and Registry change monitor module are not of the same version.. 104 Could not locate the file filename........................................... 104 Appendix E. Supported VxVM Error Conditions........................... 105 Supported VxVM Capacity and Rules Related Error Conditions................ 106 1. VxVM license.......................................................... 106 2. VxVM Disk group free space is critical.................................... 106 3. VxVM Disk free space is critical.......................................... 106 4. Read time rule for disk, subdisk, plex and volume.......................... 106 5. Write time rule for disk, subdisk, plex and volume.......................... 106 6. Read, write, wait rules for given disk, subdisk, plex and volume............ 106 Supported VxVM Standard Error Codes..................................... 107 Code:1.................................................................. 107 Code: 2................................................................. 107 Code: 3................................................................. 107 Code: 4................................................................. 108 Code: 5................................................................. 108 Code: 6................................................................. 108 Code: 7................................................................. 109 Code: 8................................................................. 109 Code: 9................................................................. 109 Code: 10................................................................ 110 Code: 11................................................................ 110 Code: 12................................................................ 111 Code: 13................................................................ 111 x Administrators Guide
Code: 14................................................................. 112 Code: 15................................................................. 112 Code: 16................................................................. 113 Code: 17................................................................. 113 Code: 18................................................................. 114 Code: 19................................................................. 114 Code: 20................................................................. 114 Code: 21................................................................. 115 Code: 23................................................................. 116 Code: 24................................................................. 118 Code: 25................................................................. 119 Code: 26................................................................. 119 Code: 27................................................................. 119 Code: 28................................................................. 120 Code: 29................................................................. 120 Code: 30................................................................. 121 Code: 31................................................................. 121 Code: 32................................................................. 121 Code: 33................................................................. 122 Code: 34................................................................. 122 Code: 35................................................................. 123 Code: 36................................................................. 123 Code: 37................................................................. 124 Code: 38................................................................. 124 Code: 39................................................................. 125 Code: 40................................................................. 125 Code: 41................................................................. 126 Code: 42................................................................. 126 Code: 43................................................................. 126 Code: 44................................................................. 127 Contents xi
Code: 45................................................................ 128 Code: 46................................................................ 128 Code: 47................................................................ 129 VM Fatal Error Messages.................................................. 130 Code: 48................................................................ 130 Code: 49................................................................ 130 Code: 50................................................................ 130 Code: 51................................................................ 131 Code: 52................................................................ 131 Code: 53................................................................ 131 Code: 54................................................................ 132 VM Kernel Warning Messages............................................. 132 Code: 106............................................................... 132 Code: 107............................................................... 133 Code: 108............................................................... 133 Code: 109............................................................... 134 Code: 110............................................................... 134 Code: 111............................................................... 134 Code: 112............................................................... 135 Code: 113............................................................... 135 Code: 114............................................................... 135 Code: 115............................................................... 136 Code: 118............................................................... 136 Code: 119............................................................... 137 Code: 120............................................................... 137 Code: 121............................................................... 137 Code: 122............................................................... 138 Code: 123............................................................... 138 Code: 124............................................................... 139 Code: 125............................................................... 139 xii Administrators Guide
VM Kernel Panic Messages................................................ 139 Code: 126................................................................ 139 Appendix F. Event IDs.................................................141 Events and Scripts........................................................ 142 VERITAS SNMP Event IDs in SNMP v2..................................... 143 abnormaljobevent........................................................143 normaljobevent.......................................................... 144 nbuextevent............................................................. 145 nbbackupfailed.......................................................... 146 dbbackupisdisabled...................................................... 147 freezingmedia............................................................ 148 suspendingmedia.........................................................149 mediarequiredforrestore................................................. 150 downeddrive............................................................ 151 exceededcleaningfrequency............................................... 152 exceededmaxmounts...................................................... 154 mountrequest............................................................ 155 nocleaningtape.......................................................... 156 zerocleaningsleft........................................................ 157 errorvxvmevent.......................................................... 158 normalvxvmevent........................................................ 159 errorvxfsevent........................................................... 160 normalvxfsevent.........................................................161 errorvcsevent............................................................ 162 normalvcsevent.......................................................... 163 logevent................................................................. 164 processevent............................................................. 165 collectorevent............................................................ 166 coldstart................................................................. 167 Contents xiii
heartbeat................................................................ 167 VERITAS SNMP Event IDs in SNMP v1..................................... 169 VRTSnbuJobEvent........................................................ 169 VRTSnbuExtEvent........................................................ 169 VRTSnbuEvents.......................................................... 169 VRTSvxvmEvent......................................................... 170 VRTSvxfsEvent.......................................................... 170 VRTSvcsEvent........................................................... 170 VRTSLogMonitor........................................................ 170 VRTSProcessMonitor..................................................... 171 VRTSCollectorMonitor.................................................... 171 VRTSeneMonitor......................................................... 171 Index.............................................................. 173 xiv Administrators Guide
Preface This preface discusses the audience and purpose of the document and describes the typographical conventions used. Audience and Purpose of this Document This manual is intended for IT professionals who are using SNMP extension to integrate VERITAS storage environment with SNMP compatible enterprise management frameworks. It is expected that the users are familiar with VERITAS core products such as NetBackup and user-specific SNMP compatible frameworks such as HP Openview Network Node Manager. 1
Conventions Conventions The following describes typographical and other conventions used in this guide. Typographic Conventions Typographic Conventions Typeface Bold fixed width Fixed width Italics Sans serif (italics) Serif (no italics) Usage Input. For example, type cd to change directories. Paths, commands, filenames, or output. For example: The default installation directory is /opt/vrtsxx. Book titles, new terms, or used for emphasis. For example: Do not ignore cautions. Placeholder text or variables. For example: Replace filename with the name of your file. Graphical user interface (GUI) objects, such as fields, menu choices, etc. For example: Enter your password in the Password field. Notes and Cautions Note Note This is a Note. Notes are used to call attention to information that makes using the product easier or helps in avoiding problems. Caution Caution This is a Caution. Cautions are used to warn about situations that could cause data loss. Key Combinations Some keyboard command sequences use two or more keys at the same time. For example, holding down the Ctrl key while pressing another key. Keyboard command sequences are indicated by connecting the keys with a plus sign. For example: Press Ctrl+t 2 Administrators Guide
Introduction 1 VERITAS SNMP Extension automatically and reliably monitors and notifies system administrators of hardware or software failures in a VERITAS Storage Management environment. If desired, it can take the corrective actions necessary to resolve the problem. VERITAS SNMP Extension offers an extensible SNMP and MIB based enterprise framework integration and intelligent proactive event management for VERITAS data availability solutions such as: NetBackup Volume Manager File System Storage Migrator Cluster Server The products listed above can be worked with out-of-box. It is also possible to integrate other VERITAS products with SNMP through VERITAS Consulting Servicees. It enables centralized frameworks to monitor, control and configure your mission-critical VERITAS environment. It intelligently monitors the VERITAS data availability environments and notifies system administrators of various events, whether hardware or software, and, if desired, automatically takes predefined corrective actions necessary to resolve the problem. SNMP events can be integrated to framework tools such as HP OpenView or sent directly to the system administrator via pager or email. The VERITAS SNMP extension allows customers to use their frameworks to manage VERITAS products. Advanced framework functions like event correlation, web based interfaces, reporting, and escalation policies can now be applied to VERITAS software products. Using the SNMP Extension, customer achieve enterprise-level data availability management, proactive event management and streamlined business process. The VERITAS SNMP Extension is available as an extension to core VERITAS software products. In addition, a web-based GUI interface and a centralized event repository are available for users to achieve integrated storage operation. 3
Description of VERITAS SNMP Extension Description of VERITAS SNMP Extension VERITAS SNMP Extension comes with a unique extensible MIB and SNMP Agent for the following VERITAS products: VERITAS NetBackup (NBU) VERITAS Cluster Server (VCS) VERITAS Volume Manager (VxVM) VERITAS File System (VxFS) VERITAS Storage Migrator (VSM) The SNMP Extension provides functionality that addresses the unique needs in each product area. Architectural Overview The VERITAS SNMP Extension employs a leading edge SNMP MIB based framework integration and intelligent proactive event management technology. It uses a multi-threaded SNMP and Event management service which bridges the framework and VERITAS products. Complete SNMP Integration 4 Administrators Guide
Architectural Overview While up and running, the service does the following: Constantly monitors the health of the VERITAS product environment Collects VERITAS product information upon framework operator s requests Automatically captures and notifies of events which are conditions that system administrators view as important to the overall functionality of their environment. Events can vary in nature and depending on the VERITAS product involved. Illustration of Proactive Event Notification Types of Events General event types include: Information and warning messages Severe or fatal error conditions Capacity related conditions (where pre-defined thresholds are exceeded) Process related conditions (detection of hanging or stopped processes) Chapter 1, Introduction 5
Architectural Overview For each supported VERITAS product, there are a number of predefined out-of-box events. You can also add non-standard events. Pre-Defined Events, per Product VERITAS Product Sample SNMP Extension Event VERITAS NetBackup All backup job related errors Tape drive down Critical backup failure Backup process stalled Any standard NetBackup error message VERITAS Cluster Server Critical HA engine and agent errors Critical resources, services, resource group failures, administrative intervention is required Fail-over failed or succeeded. All system configurations have become STALE. Unable to form cluster Agent can not start or exiting Had not available. Internal engine & agent process errors VERITAS File System All critical, error and warning VxFS event VxFS file system space has dropped below a threshold VxFS license about to expire VxFS free inodes is critical VERITAS Volume Manager All critical, error and warning VxVM event The amount of space consumed in VxVM volume exceeds a threshold VxVM Disk free space is critical VxVM Disk group free space is critical Average write time (ms) is too long for given disk, subdisk, plex and volume. Average read time (ms) is too long for given disk, subdisk, plex and volume. 6 Administrators Guide
Supported SNMP Compatible Frameworks Pre-Defined Events, per Product VERITAS Product Sample SNMP Extension Event VERITAS Storage Migrator All VSM error messages VSM warning messages VSM informational messages VSM managed file system state changed events NetBackup or VSM pending request events For every event, SNMP Extension relays necessary information such as error description, and possible corrective action to a framework console, email or pager. Predefined corrective action scripts can be kicked off immediately, enabling system administrators to automate the response to a failure condition. Supported SNMP Compatible Frameworks VERITAS SNMP Extension can be configured to work with any enterprise management framework that supports the SNMP standard. The most common framework platforms -- i.e., application and network monitoring tools -- include: HP OpenView 4.0 and above HP ITO 4.0 and above VERITAS Nerve Center IBM NetView 5.1 and above CA-Unicenter Tivoli BMC Patrol NetCool Compaq Insight Manager Supported Platforms VERITAS SNMP Extension supports the following platforms: AIX4.x,5.x (32 and 64 bit) OSF 4.x, 5.x (Tru64) Solaris2.x (both 32 and 64bit) Chapter 1, Introduction 7
List of Features HPUX11.x (both 32 and 64bit) Windows NT4 and Windows 2000 List of Features Following is the feature outline for VERITAS SNMP Extension. SNMP Integration Technology Extensible VERITAS SNMP MIB for NBU and common managed objects for VM, FS and VCS. The MIB is shipped with the SNMP package. Extensible SNMP agent which supports standard SNMP polling features such as SNMP GET/SET etc. Extensible MIB and SNMP agents. Seamless integration with all industry-leading SNMP framework, application, MIB browser etc. including HP OpenView Network Node Manager, IBM NetView, Tivoli NetView, Tivoli TEC, CA UniCenter, TNG, Micromuse Netcool/OMNIbus, BMC Patrol and VERITAS Nerve Center. An enabled framework and Web management solution to do event correlation, enable web and graph based report by taking advantage of framework build-in reporting capacity and Web SNMP management reporting capacity --- alternative solution for NBU advanced reporter. Scalable alternative solution for SNMP based management across SAN. User can manage NBU, VM, FS, VCS anywhere, anytime with a framework s web based SNMP console. Advanced SNMP security model. Reliable SNMP event using SNMP version 2c. Also support SNMP version 1. Storage Management Technology Out-of-box, hundreds of frequent error events for NBU, VCS, VM, FS, and VSM. Ability to proactively control storage component failures by Automated proactively monitor NBU etc. based on predefined policies. Automated detection. Automated correction Ability for administrators to prevent disruption by regular health checking 8 Administrators Guide
List of Features Advanced event filtering to eliminate event storm Automated responses: Notification using email, pager, reliable SNMP events, log. Kick-off of corrective UNIX or NT commands/scripts Forwarding of event, allowing SNMP framework to poll tables of information Capacity of SNMP to detect many different categories of error events based on advanced event management technology. It can detect errors using following method: Error log monitoring Any error log, any error string sequences Process monitoring Any process, availability, hanging and runaway. Threshold Policy and Collector based monitoring Querying NBU DB for attribute values such as job s kbyte info, and compare with predefined threshold. Definition of SNMP event notification using SNMP v1, SNMP v2, and SNMP v2c standards. It not only has enhanced security, but also guaranteed event delivery. Besides SNMP notification, customers also have the option to let SNMP send events directly to the management console. The management console events integration script is in the updated package too. Centralized knowledge repository and an easily accessible web front-end to allow users to do the following: Track events Perform change management Build knowledge base Generate reports Configure SNMP Extension For more information about functionalities, please email: vxsnmp@veritas.com. Chapter 1, Introduction 9
List of Features 10 Administrators Guide
Installation 2 This chapter tells how to plan for installation of VERITAS SNMP Extension and how to get and install the software. Topics covered are: System Requirements Pre-Installation Questions to Consider Where to Get VERITAS SNMP Extension Software Installing VERITAS SNMP Extension on UNIX Installing VERITAS SNMP Extension on Windows SNMP Extension Files Available After Installation Support and Distribution 11
System Requirements System Requirements SNMP supports both 32 and 64 bit on AIX4.x,5.x, Solaris2.x and HPUX11.x and Tru64 platforms and Windows platforms, including NT and Windows 2000. Supported Platforms VERITAS SNMP Extension supports the following platforms: AIX4.x,5.x (32 and 64 bit) OSF 4.x, 5.x (Tru64) Solaris2.x (both 32 and 64bit) HPUX11.x (both 32 and 64bit) Windows NT4 and Windows 2000 Required Packages You should have a VRTSene_<platform>.tar.Z package and a separate Perl package, if the system does not already have Perl installed. Platform Solaris2.x HPUX11.x AIX4.3.x, 5.1 WIN32 OSF 4.x, 5.x (Tru64) Package perl2.6.tar.z perl_hp11.tar.z perl_aix.tar.z ActivePerl perl_tru64.tar.z Note If you are working on a Windows platform, and you have not yet installed Perl, install it in a path that does not contain spaces within the name. That means that you should, for example, install it in a directory other than Program Files. 12 Administrators Guide
Pre-Installation Questions to Consider Pre-Installation Questions to Consider We recommend that you complete the following checklist before installing the VERITAS SNMP Extension. Server Information System Name: Hardware Platform: Operating System and Version: Hostid: Memory: Number of servers to be monitored: Amount of disk space available (40 MB): Is there ftp, telnet capacity or web connectivity to servers? Yes/No Is administrator, who has access to all servers, available? Yes/No SNMP Framework Which SNMP Framework and Version is being used: Is the framework currently installed and being used: Yes/No Is the framework server reachable by the VERITAS SNMP Extension servers? Yes/No Monitoring Information For each Server: System Name: Hardware Platform: Operating System and Version: VERITAS NetBackup Extension required:yes/no VERITAS File System Extension required:yes/no VERITAS Volume Manager Extension required:yes/no VERITAS Cluster Server Extension required: Yes/No Chapter 2, Installation 13
Where to Get VERITAS SNMP Extension Software VERITAS Storage Migrator: Yes/No Where to Get VERITAS SNMP Extension Software You can download the necessary files from the ftp site mfgftp.veritas.com. The subdirectory is VRTSene. Use the following login information from the command line: Username: readsnmp Password: F**dB@ck Directory: VRTSene Installing VERITAS SNMP Extension on UNIX The following example shows installation for Solaris. The steps are similar with AIX and HP-UX, Tru64. If you use SNMP for NetBackup, install on the NetBackup master server first. Then install on separate NetBackup media servers. The start up arguments for vxened are different on master and media server. Note The native installation package for Solaris and HP-UX are also available. Please contact vxsnmp@veritas.com for details. To install on a UNIX platform: 1. Login as root. 2. Copy VRTSene.tar into the installation directory. This example assumes that the directory is /disk1. 3. cd /disk1 4. tar -xvf VRTSene.tar 5. ln -s /disk1/vrtsene /usr/openv/ene 6. cd /usr/openv/ene/bin By default, the binaries run on Solaris 2.7,8 64 bits systems. If you are installing on a 32 bit system, create a symbolic link for the right binaries. For example: if this is a Solaris2.6 system, then use the following command: 14 Administrators Guide
Installing VERITAS SNMP Extension on Windows ln -s vxened.32bit vxened, ln -s getmsg.32bit getmsg 7. If you already have Perl installed, then you need not install it again. But you need to go through the following scripts in the /usr/openv/ene/config/scripts directory, to replace the perl binary path /usr/openv/ene/perl/bin/perl with the existing path, for example, /usr/bin/perl. Look through every file, including the following: ActionITO.prl, ActionTemplateITO.pl, CleanLog.prl, Dbbackup, NBUKbytes, NBUPoolTapes, NBUPoolTapes_UNIX.pl, NBUProcess, NBUextTemplate, NBUmediaList, StartMM, StartNBU, StopMM, StopNBU, bpdowndrives, bpkbyte.prl, process.pl 8. If you do not already have Perl installed, install Perl as follows: a. Login as root. b. Copy perl.tar.z into the installation directory, for example /disk2 c. cd /disk2 d. Uncompress perl.tar.z. e. tar -xvf perl2.6.tar f. ln -s /disk2/perl2.6 /usr/openv/ene/perl 9. Customize the notify.conf file and the snmpd.conf file. to change the SNMP_HOST name to the actual SNMP framework host names to make SNMP start working. Installing VERITAS SNMP Extension on Windows To install on a Windows platform: 1. Install ActivePerl on the system. Perl will be required for installing VERITAS SNMP Extension. Install Perl in a path that does not contain spaces in the name. That means that you should not install it in the Program Files directory, as is often done. If you have already installed Perl in a path that has any spaces in the name, you must either uninstall and reinstall it or create a second installation of Perl on your system. 2. Make sure your Perl setup includes support and libraries for Win32. Chapter 2, Installation 15
SNMP Extension Files Available After Installation 3. Install SNMP following the Install Shield steps. Note Do not install in the default path Program Files or in any path that contains a space in it. a. Install SNMP on the NetBackup master server. b. Install SNMP on separate NetBackup media servers. The start up arguments for vxened are different on master and media server. 4. Customize the notify.conf file and the snmpd.conf file. to change the SNMP_HOST name to the actual SNMP framework host names to make SNMP start working. (See Customizing notify.conf and Customizing snmp.conf. ) SNMP Extension Files Available After Installation SNMP extension files exist in the /usr/openv/ene directory. After installation, you should have following: VRTSene_README.doc bin directory for SNMP binary and scripts: S998ene script to start and stop vxened. Putting S998ene in /etc/rc3.d will enable automatic startup of SNMP after a reboot. On HPUX, the directory is /sbin/rc3.d. vxened: The binary for SNMP and event notification agent daemon snmptrap, snmpinform: SNMP binary utilities The scripts to send VERITAS events: jobevent nbuevent psevent logevent clevent testevent nbuextevent hbevent vcsevent 16 Administrators Guide
SNMP Extension Files Available After Installation vxvmevent vxfsevent vsmevent The utility to get NetBackup job error message, explanation, and recommended actions: getmsg Scripts to send email: On UNIX: VRTSene_send_mail.sh On WIN32: VRTSene_send_mail.pl config directory, the SNMP and Event configuration directory, which contains the following: PerlModules directory, which contains sample Perl modules. Scripts directory, which conains sample Perl scripts snmp.conf and snmpd.conf files, which define how SNMP agent and applications operate. If renamed, the SNMP thread will be disabled in vxened process. freq.conf file, which sets the message, sampling and process frequency in seconds logfile.conf configuration file for any error logfile to monitor, it is used together with messages.conf. messages.conf configuration file for monitoring NBU, VxVM,and VxFS customizable error messages. Specify any error string sequence to monitor here. collector.conf configuration file for out-of-box and add-on collectors process.conf configuration file for process monitoring. On AIX, the process monitoring is moved to collector.conf. notify.conf configuration file for notification and filtering. nbu.conf file, whose presence enables NetBackup event notification. vcs.conf, whose presence enables VCS event notification. vxfs.conf file, whose presence enables VxFS event notification. vxvm.conf file, whose presence enables VxVM event notification. vsm.conf file, whose presence enables VSM event notification. snmp directory, which contains VERITAS SNMP MIB and necessary mibs. mibs directory contains the mibs definition. data directory, which holds the temporary sampling data. Chapter 2, Installation 17
Support and Distribution doc directory logs directory. The local copy of preformated event detected will be put into logs/db/daily_messages.log. For debug information, user can create logs/vxened directory and restart the process. Note If you also want to have centralized event repository database, web graphic user interface, and event tracking capacity, please email vxsnmp@veritas.com. Support and Distribution VERITAS Customer Support, 1-800-342-0652 in the U.S., +44 (0) 870 606 6000 in the U.K. and Europe. Email vxsnmp@veritas.com for support calls and feedbacks. 18 Administrators Guide
Standard Frameworks Configuration 3 The configuration steps for standard SNMP frameworks are similar to the steps HP OpenView Network Node Manager. This chapter covers the following topics: HP OpenView Network Node Manager SNMP Event Configuration HP ITO Integration Steps IBM/Tivoli NetView or TEC Configuration Steps CA UniCenter TNG Enterprise Manager Configuration 19
HP OpenView Network Node Manager SNMP Event Configuration HP OpenView Network Node Manager SNMP Event Configuration The configuration steps for other SNMP framework are similar to HP OpenView Network Node Manager. 1. Load the VERITAS SNMP MIB file VRTS-SNMP-MIB.txt. For example in HP OpenView Network Node Manager, go to menu Options --> Load/Unload MIBs. 2. After loading the MIB, the VERITAS Events definition of Notification type will be automatically loaded into HP OpenView event configuration window. 3. Create the "Veritas NetBackup Events" category. a. Go to Edit->Configure->Event Categories to bring up the Event Configuration dialog. b. Enter Veritas NetBackup Events in the Category Name. c. Click Add. You can see the new entry in the dialog. d. Click Close to close the dialog box. 4. Modify the automatically loaded VERITAS events in the modify events dialog box s event message page. Change the actions to log and display in the category Veritas NetBackup Events. 5. Configure the community and remote port for the node where ENE and VERITAS products are installed. a. For example HP OpenView Network Node Manager, go to menu Options --> SNMP configuration to start the SNMP configuration property sheets. b. Go to Specific Nodes page. c. Click Add to get the SNMP parameter dialog box. Here you can specify community and remote port to use for SNMP polling on targeted node. 6. Modify the manager and host name in the SNMP event test script $ENEdir/bin/testevent, and run it to make sure the event is received. You can customize in OpenView NNM framework with VERITAS SNMP Extension s MIB and SNMP agent support. You can add icon, map, table, graph, or menu entries (to launch commands) etc. in HP OpenView NNM. The detailed customization steps are available from: http://ovweb.external.hp.com/lpe/doc_serv/. 20 Administrators Guide
HP ITO Integration Steps HP ITO Integration Steps 1. Perform the SNMP configurations steps in Network Node Manager first. 2. Export the translated event configuration from NNM, and load it in HP ITO. Allowing New Trap Definitions Once the SNMP trap template has been heavily customized, great care must be taken in any attempts to upload new SNMP trap definitions from trapd.conf via ovtrap2opc or opccfgupld. The following procedure is designed to allow many new trap definitions that might appear in the trapd.conf file after MIB uploads or third- party products have been integrated with NNM/ITO: 1. Make a backup of the trapd.conf file. 2. Load new SNMP Trap Definitions. Remember that many SNMP MIBs contain trap definitions that can be automatically appended to the trapd.conf file. The xnmloadmib front-end GUI for loading MIBs will prompt the user to inform if there are trap definitions associated with the MIB. Remember that installation of some third-party products may invoke xnmloadmib. 3. Check the trapd.conf file for the new trap definitions. If there are only a handful of traps, seriously consider updating the ITO SNMP trap template manually. Use the UNIX diff command if available. 4. Use the UNIX diff command or some other editing tool to manually separate out the new trap definitions from the trapd.conf file. 5. Create a file with the new trap definitions. Give it a name, for example trapd.opc. 6. Add a line to the top of this file with the text: "VERSION 3" just like the first line in trapd.conf. 7. Run the following command: $OV_BIN/OpC/utils/ovtrap2opc $OV_CONF/trapd.opc "My SNMP Traps" mytraps a. The script will ask whether the trap definitions are to be uploaded. b. Answer: no. c. The trap template name "My SNMP Traps" should be the name of the customized SNMP trap template as listed in the message source templates window. Chapter 3, Standard Frameworks Configuration 21
IBM/Tivoli NetView or TEC Configuration Steps d. The next argument is the name of template download directory that will be created (or overwritten) in the template download directory path, which is: /var/opt/ov/share/tmp/opc_appl. If this directory exists, you will be prompted to overwrite it. 8. Run the following command: $OV_BIN/OpC/opccfgupld -subentity -add mytraps 9. Check the message source templates and see if the new trap definitions appeared properly. There should be no need to restart the ITO GUI. 10. Assign the new template to local ITO server and client. 11. Make sure no other template is suppressing the SNMP events. 12. Edit and run $ENE_Home/bin/testevent to make sure ITO events are received. Please refer to http://www.fognet.com/itosnmppaper.htm for details of the integration. IBM/Tivoli NetView or TEC Configuration Steps NetView is similar to HP OpenView NNM. The difference is that NetView only support SNMP version 1. 1. Use the VERITAS SNMP version 1 MIB to perform the SNMP configuration steps that are similar to Network Node Manager. 2. Edit $ENE_Home/config/freq.conf file to set: snmptrap_version=v1 3. For NetView 6.0 or later, NetView can setup such that all NetView SNMP events and MIB can be understood in Tivoli Enterprise Console(TEC). See NetView user s documents. This is the recommended way. 4. If you are using TEC only, you need to following TEC documentation to configure TEC s SNMP adaptor to receive SNMP traps. 5. Following TEC documentation, and the VERITAS SNMP v1 Event OIDs at the end of this documentation, configure tec_snmp.oid, tec_snmp.cds files using vrts.oid and vrts.cds files. 6. Restart TEC event adaptor tecad_snmp. 7. In NetView environment: a. Run /usr/ov/bin/mib2trap VRTS-SNMP-MIBv1.txt to generate the scripts to add VERITAS event definitions to trapd.conf. b. Run the script to add the VERITAS event to trapd.conf. 22 Administrators Guide
CA UniCenter TNG Enterprise Manager Configuration c. The remaining steps are similar to HP OpenView NNM configuration. Note There is a bug of mib2trap utility. Therefore you may need to remove the comments in the MIB file. CA UniCenter TNG Enterprise Manager Configuration Edit $ENE_Home/config/freq.conf file to set: snmptrap_version=v1 If you prefer consolidated message for VERITAS SNMP Event, edit $ENE_Home/config/freq.conf file to set: snmptrap_version=ca To set up VERITAS SNMP Event recognition: 1. On the system where Enterprise Manager is running, check the Unix daemon or NT services to make sure the SNMP trap daemon or services from the operating system is not running. If they are running, stop them. 2. Run the unifstat command to check that the Event Management Daemon and SNMP trap service are both activated. a. If SNMP trap service is not activated, then run the caugui settings command to bring up the UniCenter TNG GUI for the settings. b. From Client preferences and Server preferences, change both SNMP trap Server activated setting and global setting to Yes. The default setting is usually No. c. Run unicntrl stop uni and unicntrl start uni to stop and start Enterprise Manager to get the new settings. d. Run unifstat again to check that both SNMP trap service and Event Management service are running. Example The user can use the flexibility and capacity of UniCenter TNG to define all kinds of actions after a NetBackup, VxVM, VxFS event, sent from ENE, is trapped. The following steps will only serve as an example to allow Event Console to trap the NetBackup critical Chapter 3, Standard Frameworks Configuration 23
CA UniCenter TNG Enterprise Manager Configuration backup job error, Storage Unit is not available: with Event ID.1.3.6.1.4.1.1035.0.0.0.213, and kick of a banner on the screen to display the full text contents of the event after the event is received. 1. In Event Manager, bring up the message record configuration window. 2. Click menu: Message-->New to get the Message Record Detail dialog box. a. Put * in the message id field. b. In the message description, put Veritas Events. c. The token field will be re-generated. d. Leave all other fields empty for now. The user can always customize the filtering and actions later on. e. On the scan property page,put *1.3.6.1.4.1.1035* in the Text field. If you want to trap specific events, you need to use the specific VERITAS events ids at the end of this doc to set this up. For example, if you want to track NetBackup job error regardless the error code, you can put *1.3.6.1.4.1.1035.1.1.251.0.1* in the Text field. The From and To fields should both contains 0. f. Click Ok. The message should appear in the Message Record window. 3. Open the newly created message, click Actions to get the Message Record Action Summary window. 4. Click Select--> New to bring up the Message Record Action - Detail dialog box. a. You should notice the token field contains the automatically generated token for the message just defined. b. Specify a number in the Sequence number field. Since more than one action can be defined by for a message record, the number specifies when this action should be performed relative to other actions defined for the Message Record. c. Put &TEXT in the Text field. d. Choose BANNER in the Action drop-down menu. e. Choose BLINK as Attribute, and choose Red Color. 24 Administrators Guide
CA UniCenter TNG Enterprise Manager Configuration f. Leave other fields as default. g. Leave "Active", "Evaluate" options checked. h. Click Ok to close the dialog box. 5. In the Event Management Console, type opreload command so that the new definitions can be effective. 6. Edit the /usr/openv/ene/bin/testevent command to reflect the UniCenter Host and NBU Master server name. And run the command from the ENE installation. You should see the red banner pop up on the screen with the full contents of the SNMP trap. 7. To simplify the messages, the first action sequence was to select suppression. The second was to make UniCenter display $(15:) (15th delimited piece etc.) 8. Run the $ENE_Home/bin/testevent ca script or the following command to test that the event can be received and displayed as wanted: $ENE_Home/bin/snmptrap -v 1 -M../snmp/mibs UniCenter_Host public.1.3.6.1.4.1.1035.1.1.251 Local_Host_Name 6 1 "".1.3.6.1.4.1.1035.1.1.1.1.101 s "this is a test job event" Chapter 3, Standard Frameworks Configuration 25
CA UniCenter TNG Enterprise Manager Configuration 26 Administrators Guide
Customization 4 After you install VERITAS SNMP Extension, you must customize two configuration files: notify.conf snmpd.conf In addition, you may customize others, as needed. This chapter describes the configuration files and how to customize them for your use. 27
Required Customizations Required Customizations In order to make SNMP start working, you must customize the two following files to change the SNMP_HOST name to the actual SNMP framework host names: notify.conf snmpd.conf In all the.conf files, the comments are started with the # character. Blank lines are ignored. Besides snmp.conf and snmpd.conf, all entries in other.conf files contain many fields on each line. The * is the field separator. Customizing snmpd.conf Note If you are not using the SNMPGET or SNMPSET feature, you can disable the SNMP listening thread by moving snmpd.conf or snmp.conf into the config/save subdirectory. You must customize the snmpd.conf file to specify default parameters for the SNMP agent and application. In this file, you can define where to send the SNMP agent start and stop event, and you can set access control instructions. The format includes pairs, as below: KEY Value Make sure to use the proper default COMMUNITY string and access control strings. Such values should be consistent with SNMP configuration in the framework. You can disable the SNMP thread in vxened process by renaming this file. 1. Define where to send the SNMP agent start and stop event by changing the address to match that of your host In the example below, grump is the SNMP manager host s name. The SNMP v2 event will be sent to SNMP framework, such as HP OpenView NNM host: grump trap2sink grump public Other examples: trapsink HOST [COMMUNITY [PORT]] trap2sink HOST [COMMUNITY [PORT]] informsink HOST [COMMUNITY [PORT]] 2. Set access control: 28 Administrators Guide
Required Customizations i. Change the value of the source for SNMP_HOST to your specific SNMP framework hostname. If you want to allow a sub-network SNMPGET, SNMPSET access, then uncomment out and provide the proper sub-network mask by overwriting the com2sec mynetwork entry as appropriate. sec.name source community com2sec local SNMP_HOST public com2sec mynetwork 166.98.0.0/16 public com2sec mynetwork 166.98.48.0/24 public j. Map the security names for single hostname or your subnet. In the examples, "local" is a default name for a single host. "mynetwork" is the group name for a subnet. You can use any names (not just "local" or "mynetwork") as long as you are consistent throughout the snmpd.conf file. Replace "mynetwork" with the name of your own read-write group. Replace "mynetwork" with the name of your own read-only group. The examples below give the proper format. Remember to put a # sign before any line you don't want to use and remove the # sign from any line you DO want to use.: sec.model group MyRWGroupv1 group MyRWGroupv2c group MyRWGroupusm MyRWGroupv1 groupmyrwgroupv2c groupmyrwgroupusm groupmyrogroup v1 groupmyrogroup v2c groupmyrogroup usm sec.name local local local mynetwork mynetwork mynetwork mynetwork mynetwork mynetwork Chapter 4, Customization 29
Required Customizations k. Create a view for us to let the groups have rights to: incl/excl subtree mask view all included.1 80 l. Grant the group access to the 1 view with write(or read) permissions: context sec.model sec.level match read write notif access MyRWGroup "" any noauth exact all all none #access MyROGroup"" any noauth exact all none none 3. Define NetBackup class table size. The default size is 16. SNMP loads all classes. If there are more or fewer than 16 classes defined, you can change the size, as follows: CLASS_TABLE_SIZE 16 4. Define NetBackup job table size. The default size is 16. SNMP loads all recent and current job definitions in the /usr/openv/netbackup/db/jobs directory. If you have more or fewer than 16 jobs, you can change the size. Define how many days you want to keep the job infomation in this directory by placing the job.conf file in this directory. In the job.conf file, you can specify keep_days days, as follows: Example JOB_TABLE_SIZE 16 5. Define the NetBackup extensible table size. The table is a list of scripts command that the SNMP framework can execute on the NetBackup master server. The default size is 32. There are 6 out-of-box commands listed below. You can add more commands in the table by using the template NBUextTemplate. Example NBUEXT_TABLE_SIZE 32 6. Define the NetBackup extensible table of commands. There are 8 fields: key index commandname path datafile lockfile 30 Administrators Guide
Required Customizations timeout frequency Values are expressed as follows: nbuext 1 NBUextTemplate NBUextTemplate.bat NBUextTemplate.data NBUextTemplate.lock 38 4 nbuext 2 StartNBU StartNBU.bat StartNBU.data StartNBU.lock 38 4 nbuext 3 StopNBU StopNBU.bat StopNBU.data StopNBU.lock 38 4 nbuext 4 StartMM StartMM.bat StartMM.data StartMM.lock 38 4 nbuext 5 StopMM StopMM.bat StopMM.data StopMM.lock 38 4 nbuext 6 DBBackup DBbackup.bat DBbackup.data DBbackup.lock 38 4 7. Define NetBackup media table size: MEDIA_TABLE_SIZE 32 8. Define the NetBackup media collector, the collector to inventory the media on the NBU master server. MEDIA_COLLECTOR NBUmediaList media.data media.lock 3000 100 9 Customizing notify.conf You must customize the notify.conf file to specify the method of notifiation for each object type. There are 7 fields: notifytype notifyaddr notifyseverity notifyobjtype notifyerrcode Hostname ClassName The format is as follows: notifytype*notifyaddr*notifyseverity*notifyobjtype*notifyerrcode*hostname*classname* The two fields notifytype and notifyaddr are required. Other fields are optional and can serve as filtering criteria. If the optional fields are not specified, there will be less filtering checking before the notification. The terminating * is required. If you do not use a field, you must still use its terminating asterisk as a placeholder. For example: 1*EMAIL_ADDRESS***** Chapter 4, Customization 31
Required Customizations means: 1*EMAIL_ADDRESS*EmptyField*EmptyField*EmptyField*EmptyField* 1. Type an integer to indicate notification type. notifytype is a required field. Valid values for notifytype are: EMAIL = 1 SNMP = 2 ACTIONS = 3 PAGER = 4 FILTER = 5 notifytype = 5 will only filter repetitive error codes within the time interval specified in filtering= entry in freq.conf. EXCLUDE = 7 notifytype = 7 will exclude error codes which fall into the range specified. 2. notifyaddr is a required field. It is a string which can be any of the following: The path for logfile The email address The fully qualified snmp management server address The path for action script 3. notifyobjtype is optional. It will not be used for notification filtering purposes. notifyobjtype has a valid range from 0 to 16, among which 0 through 8 are predefined as following. The rest are user defined: NBU = 0 VXVM = 1 VXFS = 2 PROCESS_N = 3 LOG_OBJ = 4 NT_EVENT = 5 COLLECTOR = 6 VCS = 7 32 Administrators Guide
Required Customizations ANY_OBJ = 8 VSM = 10 USER_DEF 9-16 If the notifyobjtype is PROCESS, then processname. hostname, state, and severity will be the script's arguments. If notifytype is PAGER, then the pager address is the absolute path of a specific script., which will be called with the following arguments: hostname, severity, process name and state. If the notifyobjtype is NBU, then Severity, HostName, HostType, ClassName, ScheduleName and ScheduleType, VolPool, STU, ErrorCode will be the arguments. Some arguments will not be available for Error Code larger than 254 4. notifyseverity is optional. Values for notifyseverity have the following meanings: Critical = 0, Major = 1, Minor = 2, Warning = 3, Normal = 4 5. notifyerrcode is optional and indicates the error code or error code ranges. Only if it is matched with the error code or range will the notification be sent. On both UNIX and Windows, the error code supports multiple range expressions such as 3-149,152-154 6. HostName is optional and is another filtering criteria for notification. When the error code is 1-254, you can specify the backup client hostname here to filter which system administrator to page. Otherwise, only if it is matched with the host where the error occurred, the notification will be sent. 7. ClassName is optional and is another filtering criteria for notification. When error code is 1-254, you can specify the backup class name to filter which system administrator is responsible for the class and should be notified. Otherwise, only if it is matched with the class name where the error occurred will the notification be sent. Examples The following is an example of email configuration: Chapter 4, Customization 33
Optional Customizations 1*Firstname.Lastname@company.com*****ClassA* #1*EMAIL_ADDRESS1_TO_SEND*****ClassA* #1*EMAIL_ADDRESS2_TO_SEND*****ClassA* The following is an example of SNMP notification. Only the events related to Class A will be sent: 2*grump.cam.company.com*****ClassA* #2*SNMP_HOST1*****ClassB* #2*SNMP_HOST2****** The following example shows Action configuration. The action scripts must be valid: 3*/usr/openv/ene/config/Scripts/Action.sh***** The following example shows pager configuration. The action scripts must be valid: 4*/usr/openv/ene/config/Scripts/PageAction.sh***** The following example shows Airtouch pager configuration: 1*8888569136@mobilecomm.com***2000** The following example shows eexternal pager configuration using scripts: 4*/usr/openv/ene/config/Scripts/PageQuan.sh***2000** The following example shows how to filter out repetitive error codes, for example, to filter out events with error code 0, 1, 150, 304. Only one such entry is allowed., is the separator. 5*0,1,150,304* The following example shows how to completely exclude code ranges. Multiple entries are allowed. For example, the following entry will exclude events with error code falling into range: 0-2, 151-165: 7****0-2,151-165** Optional Customizations Customizing snmp.conf Note If you are not using the SNMPGET or SNMPSET feature, you can disable the SNMP listening thread by moving snmpd.conf or snmp.conf into the config/save subdirectory. You can customize the snmp.conf file to specify default parameters for the SNMP agent and application, using pairs in the format: 34 Administrators Guide
Optional Customizations KEY Value Note Note that there is no equal sign (=) between Key and Value. The following default values are included. Make sure to use the proper default COMMUNITY string and default PORT number. Such values should be consistent in snmpd.conf specification and SNMP configuration in the framework. You can disable the SNMP thread in the vxened process by renaming this file. 1. In the mibdirs (mib-dirs +mib-dirs) field, specify a : separated list of directories to search for mibs. mibdirs D:/VERITAS/ENE/snmp/mibs 2. In the mibs (mib-tokens +mib-tokens) field, specify a : separated list of mibs which need to be parsed: mibs VRTS-SNMP-MIB 3. In the defaultport field, indicate the default port number that SNMP agent should use. The SNMP framework should also use the port number defined below for SNMP Polling(SET/GET) operation. defaultport 1661 4. In the defversion (1 2c 3) field, indicate the default snmp version to use. defversion 2c 5. In the defcommunity field, indicate the default snmpv1 and snmpv2c community string to use. defcommunity public Customizing freq.conf Customize this file to set the message, process, heartbeat, filtering monitoring frequency. Here you can also set the VxVM and VxFS monitoring frequency in seconds. 1. Set, in number of seconds, the interval for message log checking.we recommend it be 20 seconds: messagefreq=20 2. Set, in number of seconds, the interval for process checking. We recommend it be 300 seconds. Chapter 4, Customization 35
Optional Customizations processfreq=300 Note On AIX, the frequency is defined in collector.conf. 3. Set the heart-beat frequency, which determines the time interval for periodically sending out SNMP heartbeat event to the SNMP console. If this entry is commented out, the heart beat feature will be disabled. heartbeat=3600 4. Set the time interval to eliminate repeated events with the same error codes. Such codes are specified in the notify.conf file, starting with 5*. Only the first event will get through for the defined period below. To completely exclude error code ranges, use the excluding feature in notify.conf. filtering=86400 5. Define the snmptrap version to send to framework. For SNMPv1 Trap: use v1 For SNMPv2 Trap: use v2 To send to CA UniCenter, use ca. The default is v2. For individual SNMP v2 trap for each NBU job error code, use 2i. The MIB that needs to be loaded in the framework is: VRTS-SNMP-MIB2i.txt. For individual SNMP v1 trap for each NBU job error code, use vi. The MIB that needs to be loaded in the framework is: VRTS-SNMP-MIBvi.txt. For example: snmptrap_version =v2 6. Define the backward compatibility support: support_nbu45 = 1 (means not backward compatible) support_nbu45 = 0 (DEFAULT: means it is compatible) Always use 0 for backward compatibility. Customizing logfile.conf Configure this file to contain the list of logfiles you want SNMP to monitor. There are two fields: LogfilePath 36 Administrators Guide
Optional Customizations LogfileIndex They are expressed in the following format: LogfilePath*LogfileIndex* 1. In the LogfilePath field, indicate the absolute path of a logfile. This value is required. 2. In the LogfileIndex field, specify an index for the logfile. This value is required, and it must be greater than or equal to 2. The index is used in the messages.conf file to associate the scan text with the logfile it should search for. The index 0 is reserved for the /usr/openv/netbackup/db/error/daily_messages.log, which can be overridden by the environment variable NBU_MSG_LOG. The index 1 is reserved for the system log file /var/adm/messages on Solaris and AIX, which can be overridden by the environment variable SYS_MSG_LOG. On HP-UX, the default is /var/adm/syslog/syslog.log. The following example gives index 3 to the vault log file: /usr/openv/netbackup/vault/production/bpvault.all.output_v1*3* The following example gives index 104 to the VCS log file: /var/vrtsvcs/log/engine.log_a*104* All VCS related logfiles must use index from between 100 and 199. Index numbers between 100 and 110 are reserved for VCS engine logs. For example, to map engine log to index 101: /var/vrtsvcs/log/engine.log_a*101* Index numbers between 111 and 199 are reserved for VCS agent logs. For example, to map agent log to index 112: /var/vrtsvcs/log/ip_a.log*112* Logfile index numbers from 200 to 209 are reserved for VxVM logfiles. For example: /var/adm/configd.log*201* Logfile index numbers from 300 to 309 are reserved for VxFS logfiles. For example: /var/adm/vxfs.log*301* Logfile index numbers from 700 to 999 are reserved for VSM logfiles. For example: Chapter 4, Customization 37
Optional Customizations /tmp/hsm.log*700* Index 700 is reserved for Global VSM log file. Index 701-999 are reserved for VSM managed file system logs. Customizing messages.conf Configure this file to edit sequences of error strings to track in system log and netbackup log files as well as any log file indexed in logfile.conf. The default system log file is /var/adm/messages, and the default netbackup error log is /usr/openv/netbackup/db/error/daily_messages.log. You can also scan and trap important messages in any logfile, such as bpvault's logile, or VCS engine logfile by specifying a LogFileIndex provided in logfile.conf. There are eight fields: MsgCode ObjType SearchText Desc UseDefault Resolution Severity LogFileIndex The format of this file is as follows: MsgCode*ObjType*SearchText*Desc*UseDefault*Resolution*Severity* LogFileIndex* The terminating * is required. If you do not use a field, you must still use its terminating asterisk as a placeholder. 1. In the MsgCode field, type a positive integer error code. This value is required. 2. In the ObjType field, type an integer to indicate the object type. This value is required. On NT, the ObjType 5 will be the NT event log application source, and can only be used in NT eventlog search, that is when LogFileIndex is 1. ObjType values are as follows: NBU = 0, 38 Administrators Guide
Optional Customizations VXVM = 1, VXFS = 2, PROCESS_N = 3, LOG_OBJ = 4, NT_EVENT = 5, COLLECTOR = 6, VCS = 7, ANY_OBJ = 8, VSM = 10 In addition, you can define your own types using numbers 9 through 16. 3. In the SearchText field, type a sequence of strings separated by : to be searched in log file. This value is required. 4. In the Desc field, type the error description. This value is required. 5. In the UseDefault field, type either 0 or 1. This value is required only when the UseDefault field is 1. In this case, the resolution will be: Read [VxVM VxFS] system admin guide or major error document for the complete resolution for error code $MsgCode. 6. In the Resolution field, type a string describing the problem resolution. This value is required. 7. In the Severity field, the default Severity will be ERROR. 8. In the LogFileIndex field, type an integer indicating the file in which to search for the SearchText, specified in a previous field. This integer index and logfile map is provided in the logfile.conf file. The default is 1, which indicates the system log file, /var/adm/messages or the value from the environment variable SYS_MSG_LOG. Examples The following example will ensure that SNMP will send out an FYI when a Vault session starts and stops. The event code is 451 and 452. 451*0*START OF BPVAULT SESSION*FYI, Vault started*0*nothing to be done*4*3 Chapter 4, Customization 39
Optional Customizations 452*0*END OF BPVAULT SESSION*FYI, Vault finished*0*nothing to be done*4*3 Customizing process.conf On Windows platforms, the name of this file is services.conf. In this file, you can edit the list of process names to monitor. You can also set thresholds for the the process attributes. Note On AIX, the process existence monitoring is defined in collector.conf. Except on Solaris platforms, only the existence of the process or service is monitored. On Windows platforms, the equivalent file is services.conf, where you can specify the availability of any services to monitor. There are six fields: 0 ProcessName PercentageCpu (Must leave empty field; no longer supported) BytesIO (Must leave empty field; no longer supported) Severity Error Code MaxSleepTime The format of this file is as follows: ProcessName*[0-2]*EmptyField*EmptyField*Severity*ErrorCode* MaxSleepTime* The terminating * is required. In many cases, fields will remain empty. If you do not use a field, you must still use its terminating asterisk as a placeholder. For example: means bprd*0**** bprd*0*empty*empty*empty*empty* 1. In the ProcessName field, indicate the process for which you are testing. The following NetBackup daemon processes should be available on the master server: bpdbm bprd 40 Administrators Guide
Optional Customizations ltid vmd The following processes should be on the media server (slave server): bpbrm bptm vmd ltid txxd (tape drive) txxcd (robotic drive) 2. In the next field, type a zero to indicate that you are checking for the existence of the process specified in the previous field. The following example will monitor whether the process named bprd is missing. The default severity is ERROR, default error code is 2000. bprd*0**** If you specify 1 hanging or 2 runaway, then you must include the PercentageCpu and BlockIO fields. If 0 (existence) is specified, then Severity is optional. The default is Major. Severity levels are as follows: Critical = 0 Major = 1 Minor = 2 Warning = 3 Normal = 4 3. The PercentageCpu field must be empty. 4. The BytesIO field must be empty. 5. The MaxSleepTime field must be empty. 6. If necessary use the ErrorCode field to specify an error code. The default is 2000. The error code for the process monitoring must be greater than 400. Chapter 4, Customization 41
Optional Customizations Customizing collector.conf The the netbackup.conf file is currently the same as the collector.conf file. It is the configuration file for out-of-box and add-on collector and threshold based policies. The format and out-of-box templates are provided in the file. You can plug in any attribute collectors here to define threshold based policy. Values are expressed in key=value format. All the keys are required and must be given in the sequence below. Examples The following 3 collectors are for VERITAS NetBackup. If NetBackup is not installed on the system, these NetBackup collectors will not be used. 1. Start with the collector section name in bracket. For example: [PoolTapes] 2. Specify a valid collector script path for key "script", which should be in the $ENEDir\config\Scripts directory. For example, script=nbupooltapes 3. Use the result key to specify the file that holds the results collected from the collector above. The data file will be in the $ENEDir\data directory. Information inside the data file will be in the format objectname*attribute value*. The result key is defined as in the example below: result=pool.data 4. The collector should generate a lock file prior to the beginning of collection and remove the lock file after the collection. Use the lock key to specify the lock file path. It will be in the $ENEDir\data directory. For example, lock=pool.lock 5. Use the samplefreq key to specify how frequently, in seconds, a collector should be executed. The following says to check available tapes in all volume pools every 50 minutes. For example: samplefreq=86400 6. Use the freq key to specify how frequently SNMP should check whether the collected data is ready. The optimal value can be obtained by finding out how long a collector session is usually taking. For example: freq=400 42 Administrators Guide
Optional Customizations 7. Use the $timeout key to set a timeout limit in seconds. If there is no data after $timeout seconds, then the collecting session is aborted: timeout=1250 8. Use the object key to specify an object code for the collection target. 0 means NetBackup. For example: object=0 9. Use the objectname key to specify the object name for the collection. For example: objectname=volume pool 10. Use the attribute key to specify the attribute to be collected for above object name. For example: attribute=number of tapes 11. Use the code key to indicate the error code to be used for event notification purposes and error handling. For example: code=701 12. Use the severity key to indicate the severity of the error. 0 means critical. For example: severity=0 13. Use the resolution key to describe the desired resolution when the threshold value is violated. For example: resolution=please add more tapes to given volume pool 14. Use the low key to indicate a low threshold. The number of tapes cannot be less than 5. For example: low=5 15. Use the high key to indicate a high threshold. If high threshold is empty, that means there is no cap. For example: high= 16. Use the count key to indicate the number of objects about which we care whether the attribute value is within low and high threshold. For example: count=1 Chapter 4, Customization 43
Optional Customizations Use the names key to list the objects about which we care when the threshold is violated. Object names are separated by a semicolon (;).If you leave name empty, it is a wild card, meaning that we care about all objects. For example: names=scratch Examples The following example is the beginning of the second collector by the collector name within the brackets: [Kbytes] script=nbukbytes result=kbytes.data lock=kbytes.lock # sampling frequency of 50 minutes, or 3000 seconds: samplefreq=3000 # data usually available within 30 seconds freq=120 #if no data after 100 seconds, do not wait for timeout=400 # object code 0 means NBU object=0 objectname=backup Job attribute=kbytes # error code if threshold is violated code=702 # 0 means critical severity=0 resolution=please kill and restart the hanging backup jobs # object names here to indicate filtering when threshold is overflowed. # leave name empty make it a wild card, means any backup jobs # number of tapes cannot be less than 10 low=10 # high threshold is empty, that means no cap high= count=0 # no names, means all names= Customizing vcs.conf The existence of this file enables SNMP VCS monitoring. VCS uses Tags to indicate the severity of events: TAG_A = 0 44 Administrators Guide
Optional Customizations TAG_B = 1 TAG_C = 2 TAG_D = 3 TAG_E = 4 You can specify tag level 0-4 so that SNMP will only monitor the event with Tag level that is more critical(smaller). The default is 2 = TAG_C. To specify a tag level, type only the integer. You do not need to type the tag name. Customizing vxvm.conf The existance of the vxvm.conf file enables SNMP VxVM monitoring. You do not need to edit. The file resembles the example below: # PANIC=0 FATAL ERROR=1 ERROR=2 WARNING=3 NOTICE=4 INFORMATIONAL > 4 # Specify tag level 0-4 so that VERITAS SNMP Extension only # monitor the event with # severity level that is more critical(smaller) # Default is 4=NOTICE Customizing vxfs.conf The existance of the vxfs.conf file enables SNMP VxFS monitoring. You do not need to edit. The file resembles the example below: # Critical= 0, ERROR=1, WARNING=2, NOTICE=3, INFORMATIONAL=4 # Specify tag level 0-4 so that VERITAS SNMP Extension only # monitor the event with # severity level that is more critical(smaller) # Default is 3=NOTICE Customizing vsm.conf The existance of the vxfs.conf file enables SNMP VSM monitoring. You do not need to edit. The file resembles the example below: # Critical= 0, MAJOR=1, ERROR=2, WARNING=3, INFORMATIONAL=4 # Specify tag level 0-4 so that VERITAS SNMP Extension only # monitor the event with # severity level that is more critical(smaller) # Default is 4=INFORMATIONAL Chapter 4, Customization 45
Agent Usage Agent Usage In using agents, follow the rules below: If VxVM is installed on the host, enable the VxVM message log, and run the following as root: vxconfigd -k -x logfile=/var/adm/messages On UNIX systems, check the /etc/syslog.conf file to ensure that the following equivalent configuration line is presented: *.err;kern.debug;daemon.notice;mail.crit /var/adm/messages AIX by default does not log very much. Instead, ensure that the following entry in the /etc/syslog.conf file: *.emerg;*.alert;*.crit;*.warning;*.err;*.notice;*.info /var/adm/messages On AIX, if the /var/adm/messages file does not exist, you will have to create it. On HPUX, if the default system log is not /var/adm/messages, you can either create a symbolic link /var/adm/messages to the actual system log or set the value of the actual system log to the environment variable SYS_MSG_LOG before starting vxened. After the configuration on NetBackup master server, you can check the configuration file by running vxened with the argument -check_config. To start the SNMP agent on a NetBackup master server, move into the /usr/openv/ene/bin directory, and run:./s998ene start or vxened Enter If it is a NetBackup media server only, or if you use SNMP for VM/FS only, you start SNMP without -snmp argument: vxened -continue -process -sampling To stop the SNMP agent, move into the /usr/openv/ene/bin. Run :./S998ene stop Optionally, you can start different working threads within the vxened process by specifying a combination of arguments: vxened [-check_config ] [-continue ] [-sampling ] [-process ] [-snmp ] With no arguments, the SNMP agent starts all working threads. 46 Administrators Guide
Agent Usage With only the argument -continue, SNMP only starts the message monitoring thread. With only the argument -process, SNMP only starts the process monitoring thread. On AIX, use -sampling to monitor process. With only the argument -sampling, SNMP will only start collector based monitoring threads To enable vxened startup after reboot, put S998ene into the /etc/rc3.d directory. Use S998ene start stop to start or stop vxened. If the...$enedir/logs/db directory is present, the SNMP agent will store a local copy of error events in this directory. The example format is as follows: 01:03:05 <ERROR> *wannabe*nbu*307**has exceeded maxmounts*read NBU error guide for resolution of error code 307 01:03:06 <FYI> *wannabe*nbu*309**crib unix "no cleaning tape*read NBU error guide for resolution of error code 309 01:03:06 <EMERGENCY> *wannabe*vxvm*121**subdisk disk01-03 block 357184: Uncorrectable write error*read VxVM major error document for the complete resolution for error code 121 01:04:09 <ERROR> *wannabe*/disk1*541*freespace*current free space 649000 is violated the threshold low 0.000000 and high 0.000000*Please refer system admin guide for resolution.* Chapter 4, Customization 47
Agent Usage 48 Administrators Guide
Common Events Most Common Events The table below describes the most common events, by number. This set of common events is a subset of the list of all events in Appendix B: A 2 None of the requested files were backed up 5 The restore failed to recover the requested files 6 The backup failed to back up the requested files 7 The archive failed to back up the requested files 12-15 File operation error 25 Cannot connect on socket 41 Network connection timed out 90 Media manager received no data for backup image 91 Fatal NB media database error 96 Unable to allocate new media for backup 110 /usr/openv/netbackup/bp.conf not found 213 No storage units available for use 300 DB_backup_failed 302 Freezing_media 303 Suspending_media 305 Downed_drive 49
Condition/Threshold Based Events 306 Exceeded_cleaning_frequency 309 no_cleaning_tape Condition/Threshold Based Events Rules can be set to check the number of available tapes from any NetBackup volume pool if the value is below a predefined threshold, an event will be triggered. VERITAS SNMP Extension checks for and triggers an event if : A NetBackup job is hanging There are too many downed drives The number of unassigned tapes in a given volume pool is below a certain threshold Critical Process Based Events The availability of critical daemon processes: NetBackup bprd, bpdbm, Media Manager daemons vmd, ltid. Or any critical process which needs to be present all the time. VERITAS SNMP Extension checks for and triggers an event if a NetBackup process, such as bpsched, bptm etc. is missing. Event Samples Monitor for missing NB or VM processes Monitor for cleaning tape usage & notify when change needed Monitor for media manager availability (that each media manager is online) Monitor for suspended drives Monitor for downed drives Monitor for specific backup failure types and report specific information about each failure type and event. Monitor for OTM failure (NT Only). Monitor for suspended media. Monitor for disabled backups (user suspended) Monitor for failed network connections. 50 Administrators Guide
Event Samples Monitor for failed vaulting jobs Monitor for permission denial messages during backups (NT Only). Monitor for frozen media. Report on available tapes per user-defined thresholds at both the volume and scratch pool levels Report if user/operator cancels backup Monitor for hardware errors Monitor multi-streaming systems for all files systems processed (UNIX) Monitor UNIX and NT for open files not backed up Message records must be converted to English, not straight SNMP Monitor client specific logs and messages (NT OTM), not just master & media server Error message and pertinent related information must be sent to TNG and then to Remedy The user should have both UNIX and NT knowledge to know that our backups of MS Exchange were successful and be alerted to any exceptions. To know that our backups of MS SQL Server were successful and be alerted to any exceptions. To know that our backups of Oracle were successful and be alerted to any exceptions. Out of box errors are being detected and routed to HPOV & NetCool (then spawns off a ticket to Support staff via Remedy). Enhanced monitors include : Tape Libraries - Free slots available within a tape library, media errors, cleaning tape issues, downed drives, available media within each Master/Media server, media performance, and pending requests, Advanced Reporter (daemons, arloader issues, and log files for errors), bpvault - (any backup IDs that missed being vaulted, hung vaults, vault reports, and log files for other related issues), log & core cleaner - parses and removes cores and old log files within a NetBackup Master/Media server, Miscellaneous - NetBackup catalog backup failures, and missed NetBackup catalog backups, nightly failure reports, BCV backup failures, Oracle/Informix/SQLdatabase backup issues, scratch pool checker, streams checker, report generation for billing and Customer Web site, SNMP Extension monitor (monitors it's own health), and /var/adm/messages checking. Appendix A, Common Events 51
Event Samples 52 Administrators Guide
Supported NetBackup Error Conditions B Out of the box, VERITAS SNMP Extension monitors many different Standard NetBackup (NBU) error conditions and many condition/threshold based events. All backup job non-zero exit status (1-254) are monitored. These errors and their descriptions are listed in this document. For a subset of the most common events, see Appendix A. 53
Monitored Standard NetBackup Error Conditions and Error Codes Monitored Standard NetBackup Error Conditions and Error Codes : 1-254 abnormal_backup_exit_status (Include 254 different error codes and events) 300 DB_backup_failed 301 DB_backup_is_disabled 302 freezing_media 303 suspending_media 304 media_required_for_restore 305 downed_drive 306 exceeded_cleaning_frequency 307 exceeded_max_mounts 308 mount_request 309 no_cleaning_tape 310 zero_cleanings_left Condition/Threshold Based Events Rules can be set to check the number of available tapes from any NetBackup volume pool if the value is below a predefined threshold, an event will be triggered. VERITAS SNMP Extension checks for and triggers an event if a NetBackup job is hanging. Too many downed drives NetBackup DB size exceeded predefined threshold. 54 Administrators Guide
Critical Process Based Events Critical Process Based Events The availability of critical daemon processes: NetBackup bprd, bpdbm, Media Manager daemons vmd, ltid. Or any critical process which need to present all the time. VERITAS SNMP Extension checks for and triggers an event if a NetBackup process, such as bpsched, bptm etc. is hung or runaway based on statistical threshold data about the CPU percentage and I/O. 1-254. Name: abnormal_backup_exit_status SEVERITY Warning KEY WORD EXIT STATUS non zero [1-254] NetBackup job exit with error condition described by non-zero EXIT STATUS code. It include error code from 1 to 254. It corresponds to the backup job exit code. 300. Name: DB_backup_failed SEVERITY Severe KEY WORD database backup failed The backup of the NB database failed. Investigate why the NB database backup failed. Usually the All Log Entries Report will give you some idea as to why this failed. 301. Name: DB_backup_is_disabled SEVERITY Warning Message key word NetBackup database backup is currently disabled Appendix B, Supported NetBackup Error Conditions 55
302. Name: freezing_media Backup of the NetBackup database is currently disabled. No immediate action is required. Periodically NetBackup will disable the ability to perform a NBU database backup while it is performing house-cleaning activities. Should the NB database backup be disabled for an extended period time, review the All Log Entries report to help determine why. 302. Name: freezing_media SEVERITY Warning KEY WORD FREEZING NetBackup has issued a 'freezing media' message. A piece of media (tape or optical) has been frozen either manually by an administrator or automatically by NetBackup. NetBackup will freeze media that has been previously written in another tape format (tar, ANSI) or media previously written by the NetBackup catalog. NetBackup, by default will not overwrite these tapes unless a bp.conf entry to ALLOW_MEDIA_OVERWRITE is added, see the NetBackup Administrators Guide. Also investigate the All log Entries report and bptm logs for more information. 303. Name: suspending_media SEVERITY Warning KEY WORD SUSPEND NetBackup has issued a 'suspending media' message. NetBackup tried to use the identified media but had problems reading or writing to the tape. Investigate the cause through the All Log Entries report and / or bptm log. 56 Administrators Guide
304. Name: media_required_for_restore 304. Name: media_required_for_restore SEVERITY Warning KEY WORD Is needed for the restore Media is needed for an NBU restore operation. The event is sent via a user alarm in the action script. The media identified is required for a restore that has been initiated. The media id is also identified in the Device Management Window. Locate the required tape, place it in the library or tape drive and satisfy the tape mount request through the Device Manager window. You may also deny the request through the Device Manager window. 305. Name: downed_drive SEVERITY Severe KEY WORD DOWN The operator or Media Manager has downed a drive. A tape drive is suspected of having problems. Investigate the All Log Entries report, bptm log, Operating System messages log (/var/adm/messages) to determine the cause. NetBackup may down a drive when having trouble reading a tape, if the tape is used in another drive and also has problems, then the tape is suspended and the original downed drive is automatically brought up. 306. Name: exceeded_cleaning_frequency SEVERITY FYI KEY WORD Usage has exceeded the Cleaning Frequency Appendix B, Supported NetBackup Error Conditions 57
307. Name: exceeded_max_mounts A drive has exceeded its cleaning frequency. This should not normally occur with most TLD devices such as with STK Silo s and DLT tape drives, which automatically handle cleanings. This error message only occurs when NetBackup is handling tape cleanings manually. 307. Name: exceeded_max_mounts SEVERITY Severe KEY WORD Has exceeded maxmounts A media has exceeded its maximum number of mounts. When the media was added to NetBackups' Media Manager, a maximum number of mounts was assigned. The media has reached this limit which normally signals that it should be retired and replaced with another piece of media. Data on the tape is still tracked on the tape and expires once the retention period of the image has passed. 308. Name: mount_request SEVERITY Warning KEY WORD Mount request posted for media id A mount request is posted to the system log for a non-robotic device. It may be either for the mount itself or because the media needs 'write enabled'. This message will also be displayed in the Device Manager window. Place the tape in the required non-robotic drive and satisfy the request through the Device Manager window. Deny the request through the Device Manager window. Or remove the media from the tape drive, write enable it and return it to the tape drive. This message is normally issued for tape drives under Operator control rather than AVR or Robotic control. 58 Administrators Guide
309. Name: no_cleaning_tape 309. Name: no_cleaning_tape SEVERITY FYI KEY WORD No cleaning tape or the cleaning tape is used up There is no cleaning tape or the cleaning tape is used up. This message only occurs when NetBackup is handling tape drive cleanings and should not normally occur with most TLD devices. 310. Name: zero_cleanings_left SEVERITY FYI KEY WORD 0 cleanings left The cleaning tape has no cleanings left. This message only occurs when NetBackup is handling tape drive cleanings and the cleaning tape(s) have been used to their capacity. This should not normally occur with most TLD devices. For More Information To find out more about the VERITAS NetBackup Event Notification Extension or other Advanced Solution Services offered through the VERITAS Consulting Services Group, please contact vxsnmp@veritas.com. Appendix B, Supported NetBackup Error Conditions 59
310. Name: zero_cleanings_left 60 Administrators Guide
Supported VxFS Error Conditions C VERITAS SNMP Extension natively supports fifty eight VxFS error codes and many out of box capacity and rules templates to track storage health and error conditions. Users can add new rules to check specific file system attributes against predefined thresholds. This document contains the list of error codes and provides a brief description for each. 61
Supported VxFS Capacity and Rules Related Error Conditions Supported VxFS Capacity and Rules Related Error Conditions 1. VxFS file system full The amount of space consumed in VxVM volume and its VxFS file system exceeds a threshold. 2. VxFS Defragmentation The fragmentation is deteriorated beyond a threshold. 3. VxFS Damaged 4. VxFS license File system detected to be damaged. VxFS license about to expire. 5. VxFS free inodes is critical A file system s number of free inodes is below predefined threshold. Supported VxFS Standard Error Codes Code: 001 NOTICE: msgcnt x: vxfs: mesg 001: vx_nospace - mount_point file system full (n block extent) 62 Administrators Guide
Code: 002 Code: 002 The file system is out of space. Often, there is plenty of space and one runaway process used up all the remaining free space. In other cases, the available free space becomes fragmented and unusable for some files. Monitor the free space in the file system and prevent it from becoming full. If a runaway process has used up all the space, stop that process, find the files created by the process, and remove them. If the file system is out of space, remove files, defragment, or expand the file system. To remove files, use the find command to locate the files that are to be removed. To get the most space with the least amount of work, remove large files or file trees that are no longer needed. To defragment or expand the file system, use fsadm (see the fsadm(1m) manual page). WARNING: msgcnt x: vxfs: mesg 002: vx_snap_strategy - mount_point file system write attempt to read-only file system WARNING: msgcnt x: vxfs: mesg 002: vx_snap_copyblk - mount_point file system write attempt to read-only file system The kernel tried to write to a read-only file system. This is an unlikely problem, but if it occurs, the file system is disabled. The file system was not written, so no action is required. Report this as a bug to your customer support organization. Code: 003, 004, 005 WARNING: msgcnt x: vxfs: mesg 003: vx_mapbad - mount_point file system free extent bitmap in au aun marked bad WARNING: msgcnt x: vxfs: mesg 004: vx_mapbad - mount_point file system free inode bitmap in au aun marked bad WARNING: msgcnt x: vxfs: mesg 005: vx_mapbad - mount_point file system inode extended operation bitmap in au aun marked bad If there is an I/O failure while writing a bitmap, the map is marked bad. Appendix C, Supported VxFS Error Conditions 63
Code: 006, 007 Code: 006, 007 Code: 008, 009 The kernel considers the maps to be invalid, so does not do any more resource allocation from maps. This situation can cause the file system to report out of space or out of inode error messages even though df may report an adequate amount of free space. This error may also occur due to bitmap inconsistencies. If a bitmap fails a consistency check, or blocks are freed that are already free in the bitmap, the file system has been corrupted. This may have occurred because a user or process wrote directly to the device or used fsdb to change the file system. The VX_FULLFSCK flag is set. If the map that failed was a free extent bitmap, and the VX_FULLFSCK flag can t be set, then the file system is disabled. Check the console log for I/O errors. If the problem is a disk failure, replace the disk. If the problem is not related to an I/O failure, find out how the disk became corrupted. If no user or process was writing to the device, report the problem to your customer support organization. Unmount the file system and use fsck to run a full structural check. WARNING: msgcnt x: vxfs: mesg 006: vx_sumupd - mount_point file system summary update in au aun failed WARNING: msgcnt x: vxfs: mesg 007: vx_sumupd - mount_point file system summary update in inode au aun failed An I/O error occurred while writing the allocation unit or inode allocation unit bitmap summary to disk. This sets the VX_FULLFSCK flag on the file system. If the VX_FULLFSCK flag can t be set, the file system is disabled. Check the console log for I/O errors. If the problem was caused by a disk failure, replace the disk before the file system is mounted for write access, and use fsck to run a full structural check. WARNING: msgcnt x: vxfs: mesg 008: vx_direrr - mount_point file system inode inumber block blkno error errno WARNING: msgcnt x: vxfs: mesg 009: vx_direrr - mount_point file system inode inumber immediate directory error errno 64 Administrators Guide
Code: 010 Code: 010 Code: 011 A directory operation failed in an unexpected manner. The mount point, inode, and block number identify the failing directory. If the inode is an immediate directory, the directory entries are stored in the inode, so no block number is reported. If the error is ENOENT or ENOTDIR, an inconsistency was detected in the directory block. This inconsistency could be a bad free count, a corrupted hash chain, or any similar directory structure error. If the error is EIO or ENXIO, an I/O failure occurred while reading or writing the disk block. The VX_FULLFSCK flag is set in the super-block so that fsck will do a full structural check the next time it is run. Check the console log for I/O errors. If the problem was caused by a disk failure, replace the disk before the file system is mounted for write access. Unmount the file system and use fsck to run a full structural check. WARNING: msgcnt x: vxfs: mesg 010: vx_ialloc - mount_point file system inode inumber not free When the kernel allocates an inode from the free inode bitmap, it checks the mode and link count of the inode. If either is non-zero, the free inode bitmap or the inode list is corrupted. The VX_FULLFSCK flag is set in the super-block so that fsck will do a full structural check the next time it is run. Unmount the file system and use fsck to run a full structural check. NOTICE: msgcnt x: vxfs: mesg 011: vx_noinode - mount_point file system out of inodes The file system is out of inodes. Monitor the free inodes in the file system. If the file system is getting full, create more inodes either by removing files or by expanding the file system. File system resizing is described in Chapter 1, The VERITAS File System, and in the fsadm(1m) online manual page. Appendix C, Supported VxFS Error Conditions 65
Code: 012 Code: 012 Code: 013 Code: 014 WARNING: msgcnt x: vxfs: mesg 012: vx_iget - mount_point file system invalid inode number inumber When the kernel tries to read an inode, it checks the inode number against the valid range. If the inode number is out of range, the data structure that referenced the inode number is incorrect and must be fixed. The VX_FULLFSCK flag is set in the super-block so that fsck will do a full structural check the next time it is run. Unmount the file system and use fsck to run a full structural check. WARNING: msgcnt x: vxfs: mesg 013: vx_iposition - mount_point file system inode inumber invalid inode list extent For a Version 2 and above disk layout, the inode list is dynamically allocated. When the kernel tries to read an inode, it must look up the location of the inode in the inode list file. If the kernel finds a bad extent, the inode can t be accessed. All of the inode list extents are validated when the file system is mounted, so if the kernel finds a bad extent, the integrity of the inode list is questionable. This is a very serious error. The VX_FULLFSCK flag is set in the super-block and the file system is disabled. Unmount the file system and use fsck to run a full structural check. WARNING: msgcnt x: vxfs: mesg 014: vx_iget - inode table overflow All the system in-memory inodes are busy and an attempt was made to use a new inode. Look at the processes that are running and determine which processes are using inodes. If it appears there are runaway processes, they might be tying up the inodes. If the system load appears normal, increase the vxfs_ninode parameter in the kernel (see Internal Inode Table Size in Chapter 5, Performance and Tuning ). 66 Administrators Guide
Code: 015 Code: 015 Code: 016 Code: 017 WARNING: msgcnt x: vxfs: mesg 015: vx_ibadinactive - mount_point file system can t mark inode inumber bad WARNING: msgcnt x: vxfs: mesg 015: vx_ilisterr - mount_point file system can t mark inode inumber bad An attempt to mark an inode bad on disk, and the super-block update to set the VX_FULLFSCK flag, failed. This indicates that a catastrophic disk error may have occurred since both an inode list block and the super-block had I/O failures. The file system is disabled to preserve file system integrity. Unmount the file system and use fsck to run a full structural check. Check the console log for I/O errors. If the disk failed, replace it before remounting the file system. WARNING: msgcnt x: vxfs: mesg 016: vx_ilisterr - mount_point file system error reading inode inumber An I/O error occurred while reading the inode list. The VX_FULLFSCK flag is set. Check the console log for I/O errors. If the problem was caused by a disk failure, replace the disk before the file system is mounted for write access. Unmount the file system and use fsck to run a full structural check. WARNING: msgcnt x: vxfs: mesg 017: vx_attr_getblk - mount_point file system inode inumber marked bad WARNING: msgcnt x: vxfs: mesg 017: vx_attr_iget - mount_point file system inode inumber marked bad WARNING: msgcnt x: vxfs: mesg 017: vx_attr_indadd - mount_point file system inode inumber marked bad WARNING: msgcnt x: vxfs: mesg 017: vx_attr_indtrunc - mount_point file system inode inumber marked bad WARNING: msgcnt x: vxfs: mesg 017: vx_attr_iremove - mount_point file system inode inumber marked bad Appendix C, Supported VxFS Error Conditions 67
Code: 017 WARNING: msgcnt x: vxfs: mesg 017: vx_bmap - mount_point file system inode inumber marked bad WARNING: msgcnt x: vxfs: mesg 017: vx_bmap_indirect_ext4 - mount_point file system inode inumber marked bad WARNING: msgcnt x: vxfs: mesg 017: vx_delbuf_flush - mount_point file system inode inumber marked bad WARNING: msgcnt x: vxfs: mesg 017: vx_dio_iovec - mount_point file system inode inumber marked bad WARNING: msgcnt x: vxfs: mesg 017: vx_dirbread - mount_point file system inode inumber marked bad WARNING: msgcnt x: vxfs: mesg 017: vx_dircreate - mount_point file system inode inumber marked bad WARNING: msgcnt x: vxfs: mesg 017: vx_dirlook - mount_point file system inode inumber marked bad WARNING: msgcnt x: vxfs: mesg 017: vx_doextop_iau - mount_point file system inode inumber marked bad WARNING: msgcnt x: vxfs: mesg 017: vx_doextop_now - mount_point file system inode inumber marked bad WARNING: msgcnt x: vxfs: mesg 017: vx_do_getpage - mount_point file system inode inumber marked bad WARNING: msgcnt x: vxfs: mesg 017: vx_enter_ext4 - mount_point file system inode inumber marked bad WARNING: msgcnt x: vxfs: mesg 017: vx_exttrunc - mount_point file system inode inumber marked bad WARNING: msgcnt x: vxfs: mesg 017: vx_get_alloc - mount_point file system inode inumber marked bad WARNING: msgcnt x: vxfs: mesg 017: vx_ilisterr - mount_point file system inode inumber marked bad WARNING: msgcnt x: vxfs: mesg 017: vx_ilock - mount_point file system inode inumber marked bad WARNING: msgcnt x: vxfs: mesg 017: vx_indtrunc - mount_point file system inode inumber marked bad WARNING: msgcnt x: vxfs: mesg 017: vx_iread - mount_point file system inode inumber marked bad WARNING: msgcnt x: vxfs: mesg 017: vx_iremove - mount_point file system inode inumber marked bad 68 Administrators Guide
Code: 017 WARNING: msgcnt x: vxfs: mesg 017: vx_iremove_attr - mount_point file system inode inumber marked bad WARNING: msgcnt x: vxfs: mesg 017: vx_logwrite_flush - mount_point file system inode inumber marked bad WARNING: msgcnt x: vxfs: mesg 017: vx_oltmount_iget - mount_point file system inode inumber marked bad WARNING: msgcnt x: vxfs: mesg 017: vx_overlay_bmap - mount_point file system inode inumber marked bad WARNING: msgcnt x: vxfs: mesg 017: vx_readnomap - mount_point file system inode inumber marked bad WARNING: msgcnt x: vxfs: mesg 017: vx_reorg_trunc - mount_point file system inode inumber marked bad WARNING: msgcnt x: vxfs: mesg 017: vx_stablestore - mount_point file system inode inumber marked bad WARNING: msgcnt x: vxfs: mesg 017: vx_tranitimes - mount_point file system inode inumber marked bad WARNING: msgcnt x: vxfs: mesg 017: vx_trunc - mount_point file system inode inumber marked bad WARNING: msgcnt x: vxfs: mesg 017: vx_write_alloc2 - mount_point file system inode inumber marked bad WARNING: msgcnt x: vxfs: mesg 017: vx_write_default - mount_point file system inode inumber marked bad WARNING: msgcnt x: vxfs: mesg 017: vx_zero_alloc - mount_point file system inode inumber marked bad When inode information is no longer dependable, the kernel marks it bad on disk. The most common reason for marking an inode bad is a disk I/O failure. If there is an I/O failure in the inode list, on a directory block, or an indirect address extent, the integrity of the data in the inode, or the data the kernel tried to write to the inode list, is questionable. In these cases, the disk driver prints an error message and one or more inodes are marked bad. The kernel also marks an inode bad if it finds a bad extent address, invalid inode fields, or corruption in directory data blocks during a validation check. A validation check failure indicates the file system has been corrupted. This usually occurs because a user or process has written directly to the device or used fsdb to change the file system. The VX_FULLFSCK flag is set in the super-block so fsck will do a full structural check the next time it is run. Appendix C, Supported VxFS Error Conditions 69
Code: 019 Code: 019 Code: 020 Code: 021 Check the console log for I/O errors. If the problem is a disk failure, replace the disk. If the problem is not related to an I/O failure, find out how the disk became corrupted. If no user or process is writing to the device, report the problem to your customer support organization. In either case, unmount the file system and use fsck to run a full structural check. WARNING: msgcnt x: vxfs: mesg 019: vx_log_add - mount_point file system log overflow Log ID overflow. When the log ID reaches VX_MAXLOGID (approximately one billion by default), a flag is set so the file system resets the log ID at the next opportunity. If the log ID has not been reset, when the log ID reaches VX_DISLOGID (approximately VX_MAXLOGID plus 500 million by default), the file system is disabled. Since a log reset will occur at the next 60 second sync interval, this should never happen. Unmount the file system and use fsck to run a full structural check. WARNING: msgcnt x: vxfs: mesg 020: vx_logerr - mount_point file system log error errno Intent log failed. The kernel will try to set the VX_FULLFSCK and VX_LOGBAD flags in the super-block to prevent running a log replay. If the super-block can t be updated, the file system is disabled. Unmount the file system and use fsck to run a full structural check. Check the console log for I/O errors. If the disk failed, replace it before remounting the file system. WARNING: msgcnt x: vxfs: mesg 021: vx_fs_init - mount_point file system validation failure When a VERITAS File System is mounted, the structure is read from disk. If the file system is marked clean, the structure is correct and the first block of the intent log is cleared. If there is any I/O problem or the structure is inconsistent, the kernel sets the 70 Administrators Guide
Code: 022 Code: 022 Code: 023 VX_FULLFSCK flag and the mount fails. If the error isn t related to an I/O failure, this may have occurred because a user or process has written directly to the device or used fsdb to change the file system. Check the console log for I/O errors. If the problem is a disk failure, replace the disk. If the problem is not related to an I/O failure, find out how the disk became corrupted. If no user or process is writing to the device, report the problem to your customer support organization. In either case, unmount the file system and use fsck to run a full structural check. WARNING: msgcnt x: vxfs: mesg 022: vx_mountroot - root file system remount failed The remount of the root file system failed. The system will not be usable if the root file system can t be remounted for read/write access. When a VERITAS root file system is first mounted, it is mounted for read-only access. After fsck is run, the file system is remounted for read/write access. The remount fails if fsck completed a resize operation or modified a file that was opened before the fsck was run. It also fails if an I/O error occurred during the remount. Usually, the system halts or reboots automatically. Reboot the system. The system either remounts the root cleanly or runs a full structural fsck and remounts cleanly. If the remount succeeds, no further action is necessary. Check the console log for I/O errors. If the disk has failed, replace it before the file system is mounted for write access. If the system won t come up and a full structural fsck hasn t been run, reboot the system on a backup root and manually run a full structural fsck. If the problem persists after the full structural fsck and there are no I/O errors, contact your customer support organization. WARNING: msgcnt x: vxfs: mesg 023: vx_unmountroot - root file system is busy and can t be unmounted cleanly There were active files in the file system and they caused the unmount to fail. When the system is halted, the root file system is unmounted. This happens occasionally when a process is hung and it can t be killed before unmounting the root. Appendix C, Supported VxFS Error Conditions 71
Code: 024 Code: 024 Code: 025 Code: 026 fsck will run when the system is rebooted. It should clean up the file system. No other action is necessary. If the problem occurs every time the system is halted, determine the cause and contact your customer support organization. WARNING: msgcnt x: vxfs: mesg 024: vx_cutwait - mount_point file system current usage table update error Update to the current usage table (CUT) failed. For a Version 2 disk layout, the CUT contains a fileset version number and total number of blocks used by each fileset. The VX_FULLFSCK flag is set in the super-block. If the super-block can t be written, the file system is disabled. Unmount the file system and use fsck to run a full structural check. WARNING: msgcnt x: vxfs: mesg 025: vx_wsuper - mount_point file system superblock update failed An I/O error occurred while writing the super-block during a resize operation. The file system is disabled. Unmount the file system and use fsck to run a full structural check. Check the console log for I/O errors. If the problem is a disk failure, replace the disk before the file system is mounted for write access. WARNING: msgcnt x: vxfs: mesg 026: vx_snap_copyblk - mount_point primary file system read error Snapshot file system error. When the primary file system is written, copies of the original data must be written to the snapshot file system. If a read error occurs on a primary file system during the copy, any snapshot file system that doesn t already have a copy of the data is out of date and must be disabled. 72 Administrators Guide
Code: 027 Code: 027 Code: 028 Code: 029, 030 An error message for the primary file system prints. Resolve the error on the primary file system and rerun any backups or other applications that were using the snapshot that failed when the error occurred. WARNING: msgcnt x: vxfs: mesg 027: vx_snap_bpcopy - mount_point snapshot file system write error A write to the snapshot file system failed. As the primary file system is updated, copies of the original data are read from the primary file system and written to the snapshot file system. If one of these writes fails, the snapshot file system is disabled. Check the console log for I/O errors. If the disk has failed, replace it. Resolve the error on the disk and rerun any backups or other applications that were using the snapshot that failed when the error occurred. WARNING: msgcnt x: vxfs: mesg 028: vx_snap_alloc - mount_point snapshot file system out of space The snapshot file system ran out of space to store changes. During a snapshot backup, as the primary file system is modified, the original data is copied to the snapshot file system. This error can occur if the snapshot file system is left mounted by mistake, if the snapshot file system was given too little disk space, or the primary file system had an unexpected burst of activity. The snapshot file system is disabled. Make sure the snapshot file system was given the correct amount of space. If it was, determine the activity level on the primary file system. If the primary file system was unusually busy, rerun the backup. If the primary file system is no busier than normal, move the backup to a time when the primary file system is relatively idle or increase the amount of disk space allocated to the snapshot file system. Rerun any backups that failed when the error occurred. WARNING: msgcnt x: vxfs: mesg 029: vx_snap_getbp - mount_point snapshot file system block map write error WARNING: msgcnt x: vxfs: mesg 030: vx_snap_getbp - mount_point snapshot file system block map read error Appendix C, Supported VxFS Error Conditions 73
Code: 031 Code: 031 Code: 032 Code: 033 During a snapshot backup, each snapshot file system maintains a block map on disk. The block map tells the snapshot file system where data from the primary file system is stored in the snapshot file system. If an I/O operation to the block map fails, the snapshot file system is disabled. Check the console log for I/O errors. If the disk has failed, replace it. Resolve the error on the disk and rerun any backups that failed when the error occurred. WARNING: msgcnt x: vxfs: mesg 031: vx_disable - mount_point file system disabled File system disabled, preceded by a message that specifies the reason. This usually indicates a serious disk problem. Unmount the file system and use fsck to run a full structural check. If the problem is a disk failure, replace the disk before the file system is mounted for write access. WARNING: msgcnt x: vxfs: mesg 032: vx_disable - mount_point snapshot file system disabled Snapshot file system disabled, preceded by a message that specifies the reason. Unmount the snapshot file system, correct the problem specified by the message, and rerun any backups that failed due to the error. WARNING: msgcnt x: vxfs: mesg 033: vx_check_badblock - mount_point file system had an I/O error, setting VX_FULLFSCK 74 Administrators Guide
Code: 034 Code: 034 Code: 035 Code: 036 When the disk driver encounters an I/O error, it sets a flag in the super-block structure. If the flag is set, the kernel will set the VX_FULLFSCK flag as a precautionary measure. Since no other error has set the VX_FULLFSCK flag, the failure probably occurred on a data block. Unmount the file system and use fsck to run a full structural check. Check the console log for I/O errors. If the problem is a disk failure, replace the disk before the file system is mounted for write access. WARNING: msgcnt x: vxfs: mesg 034: vx_resetlog - mount_point file system can t reset log The kernel encountered an error while resetting the log ID on the file system. This happens only if the super-block update or log write encountered a device failure. The file system is disabled to preserve its integrity. Unmount the file system and use fsck to run a full structural check. Check the console log for I/O errors. If the problem is a disk failure, replace the disk before the file system is mounted for write access. WARNING: msgcnt x: vxfs: mesg 035: vx_inactive - mount_point file system inactive of locked inode inumber VOP_INACTIVE was called for an inode while the inode was being used. This should never happen, but if it does, the file system is disabled. Unmount the file system and use fsck to run a full structural check. Report as a bug to your customer support organization. WARNING: msgcnt x: vxfs: mesg 036: vx_lctbad - mount_point file system link count table lctnumber bad Appendix C, Supported VxFS Error Conditions 75
Code: 037 Code: 037 Code: 038 Update to the link count table (LCT) failed. For a Version 2 and above disk layout, the LCT contains the link count for all the structural inodes. The VX_FULLFSCK flag is set in the super-block. If the super-block can t be written, the file system is disabled. Unmount the file system and use fsck to run a full structural check. WARNING: msgcnt x: vxfs: mesg 037: vx_metaioerr - file system meta data error A read or a write error occurred while accessing file system metadata. The full fsck flag on the file system was set. The message specifies whether the disk I/O that failed was a read or a write. File system metadata includes inodes, directory blocks, and the file system log. If the error was a write error, it is likely that some data was lost. This message should be accompanied by another file system message describing the particular file system metadata affected, as well as a message from the disk driver containing information about the disk I/O error. Resolve the condition causing the disk error. If the error was the result of a temporary condition (such as accidentally turning off a disk or a loose cable), correct the condition. Check for loose cables, etc. Unmount the file system and use fsck to run a full structural check (possibly with loss of data). In case of an actual disk error, if it was a read error and the disk driver remaps bad sectors on write, it may be fixed when fsck is run since fsck is likely to rewrite the sector with the read error. In other cases, you replace or reformat the disk drive and restore the file system from backups. Consult the documentation specific to your system for information on how to recover from disk errors. The disk driver should have printed a message that may provide more information. WARNING: msgcnt x: vxfs: mesg 038: vx_dataioerr - file system file data error A read or a write error occurred while accessing file data. The message specifies whether the disk I/O that failed was a read or a write. File data includes data currently in files and free blocks. If the message is printed because of a read or write error to a file, another message that includes the inode number of the file will print. The message may be printed as the result of a read or write error to a free block, since some operations allocate an extent and immediately perform I/O to it. If the I/O fails, the extent is freed and the operation fails. The message is accompanied by a message from the disk driver regarding the disk I/O error. 76 Administrators Guide
Code: 039 Code: 039 Code: 040 Resolve the condition causing the disk error. If the error was the result of a temporary condition (such as accidentally turning off a disk or a loose cable), correct the condition. Check for loose cables, etc. If any file data was lost, restore the files from backups. Determine the file names from the inode number (see the ncheck(1m) manual page for more information.) If an actual disk error occurred, make a backup of the file system, replace or reformat the disk drive, and restore the file system from the backup. Consult the documentation specific to your system for information on how to recover from disk errors. The disk driver should have printed a message that may provide more information. WARNING: msgcnt x: vxfs: mesg 039: vx_writesuper - file system super-block write error An attempt to write the file system super block failed due to a disk I/O error. If the file system was being mounted at the time, the mount will fail. If the file system was mounted at the time and the full fsck flag was being set, the file system will probably be disabled and Message 031 will also be printed. If the super-block was being written as a result of a sync operation, no other action is taken. Resolve the condition causing the disk error. If the error was the result of a temporary condition (such as accidentally turning off a disk or a loose cable), correct the condition. Check for loose cables, etc. Unmount the file system and use fsck to run a full structural check. If an actual disk error occurred, make a backup of the file system, replace or reformat the disk drive, and restore the file system from backups. Consult the documentation specific to your system for information on how to recover from disk errors. The disk driver should have printed a message that may provide more information. WARNING: msgcnt x: vxfs: mesg 040: vx_dqbad - mount_point file system quota file update error for id id. An update to the user quotas file failed for the user ID. The quotas file keeps track of the total number of blocks and inodes used by each user, and also contains soft and hard limits for each user ID. The VX_FULLFSCK flag is set in the super-block. If the super-block cannot be written, the file system is disabled. Appendix C, Supported VxFS Error Conditions 77
Code: 041 Code: 041 Code: 042 Code: 043 Unmount the file system and use fsck to run a full structural check. Check the console log for I/O errors. If the disk has a hardware failure, it should be repaired before the file system is mounted for write access. WARNING: msgcnt x: vxfs: mesg 041: vx_dqget - mount_point file system user quota file can't read quota for id id A read of the user quotas file failed for the uid. The quotas file keeps track of the total number of blocks and inodes used by each user, and contains soft and hard limits for each user ID. The VX_FULLFSCK flag is set in the super-block. If the super-block cannot be written, the file system is disabled. Unmount the file system and use fsck to run a full structural check. Check the console log for I/O errors. If the disk has a hardware failure, it should be repaired before the file system is mounted for write access. WARNING: msgcnt x: vxfs: mesg 042: vx_bsdquotaupdate - mount_point file system user id disk limit reached. The hard limit on blocks was reached. Further attempts to allocate blocks for files owned by the user will fail. Remove some files to free up space. WARNING: msgcnt x: vxfs: mesg 043: vx_bsdquotaupdate - mount_point file system user id disk quota exceeded too long The soft limit on blocks was exceeded continuously for longer than the soft quota time limit. Further attempts to allocate blocks for files will fail. Remove some files to free up space. 78 Administrators Guide
Code: 044 Code: 044 Code: 045 Code: 046 Code: 047 WARNING: msgcnt x: vxfs: mesg 044: vx_bsdquotaupdate - mount_point file system user id disk quota exceeded. The soft limit on blocks is exceeded. The soft limit can be exceeded for a certain amount of time before allocations begin to fail. Once the soft quota time limit has expired, further attempts to allocate blocks for files will fail. Remove some files to free up space. WARNING: msgcnt x: vxfs: mesg 045: vx_bsdiquotaupdate - mount_point file system user id inode limit reached. The hard limit on inodes was exceeded. Further attempts to create files owned by the user will fail. Remove some files to free inodes. WARNING: msgcnt x: vxfs: mesg 046: vx_bsdiquotaupdate - mount_point file system user id inode quota exceeded too long The soft limit on inodes has been exceeded continuously for longer than the soft quota time limit. Further attempts to create files owned by the user will fail. Remove some files to free inodes. WARNING: msgcnt x: vxfs: mesg 047: vx_bsdiquotaupdate - warning: mount_point file system user id inode quota exceeded Appendix C, Supported VxFS Error Conditions 79
Code: 048, 049 Code: 048, 049 Code: 050 Code: 051 The soft limit on inodes was exceeded. The soft limit can be exceeded for a certain amount of time before attempts to create new files begin to fail. Once the time limit has expired, further attempts to create files owned by the user will fail. Remove some files to free inodes. WARNING: msgcnt x: vxfs: mesg 048: vx_dqread - warning: mount_point file system external user quota file read failed WARNING: msgcnt x: vxfs: mesg 049: vx_dqwrite - warning: mount_point file system external user quota file write failed. To maintain reliable usage counts, VxFS maintains the user quotas file as a structural file in the structural fileset. These files are updated as part of the transactions that allocate and free blocks and inodes. For compatibility with the quota administration utilities, VxFS also supports the standard user visible quota files. When quotas are turned off, synced, or new limits are added, VxFS tries to update the external quota files. When quotas are enabled, VxFS tries to read the quota limits from the external quotas file. If these reads or writes fail, the external quotas file is out of date. Determine the reason for the failure on the external quotas file and correct it. Recreate the quotas file. WARNING: msgcnt x: vxfs: mesg 050: vx_ldlogwrite - mount_point file system log write failed A write to VERITAS QuickLog log failed. This marks the log bad and sets the full file system check flag in the super block. No immediate action required. When the file system is unmounted, run a full file system check using fsck before mounting it again. WARNING: msgcnt x: vxfs: mesg 051: vx_ldlog_start - mount_point file system log start failed 80 Administrators Guide
Code: 052 Code: 052 Code: 053 Code: 054 vx_ldlog_start failed. QuickLog logging is disabled and file system continues to use its own log. No corrective action required on the file system. Determine why the log didn t start and do administrative tasks on QuickLog (see Chapter 9, VERITAS QuickLog ). WARNING: msgcnt x: vxfs: mesg 052: vx_ldlog_stop - mount_point file system log stop failed QuickLog copies the log back to the file system after stopping logging activity. If the stop failed, VxFS treats the failure as the log going bad. No immediate action required. When the file system is unmounted, run a full file system check using fsck before mounting it again. WARNING: msgcnt x: vxfs: mesg 053: vx_ldlog_suspend - mount_point file system log suspend failed: When the file system is frozen, QuickLog is suspended; it is activated again on thaw. If this operation fails, the kernel marks the log bad and sets the full file system check flag in the super block. No immediate action required. When the file system is unmounted, run a full file system check using fsck before mounting it again. WARNING: msgcnt x: vxfs: mesg 054: vx_ldlog_resume - mount_point file system log resume failed: When the file system is thawed, QuickLog must be resumed. If this operation fails, the kernel marks the log bad and sets the full file system check flag in the super block. Appendix C, Supported VxFS Error Conditions 81
Code: 056 Code: 056 Code: 057 No immediate action required. When the file system is unmounted, run a full file system check using fsck before mounting it again. WARNING: msgcnt x: vxfs: mesg 056: vx_mapbad - mount_point file system extent allocation unit state bitmap number number marked bad If there is an I/O failure while writing a bitmap, the map is marked bad. The kernel considers the maps to be invalid, so does not do any more resource allocation from maps. This situation can cause the file system to report out of space or out of inode error messages even though df may report an adequate amount of free space. This error may also occur due to bitmap inconsistencies. If a bitmap fails a consistency check, or blocks are freed that are already free in the bitmap, the file system has been corrupted. This may have occurred because a user or process wrote directly to the device or used fsdb to change the file system. The VX_FULLFSCK flag is set. If the VX_FULLFSCK flag can t be set, the file system is disabled. Check the console log for I/O errors. If the problem is a disk failure, replace the disk. If the problem is not related to an I/O failure, find out how the disk became corrupted. If no user or process was writing to the device, report the problem to your customer support organization. Unmount the file system and use fsck to run a full structural check. WARNING: msgcnt x: vxfs: mesg 057: vx_esum_bad - mount_point file system extent allocation unit summary number number marked bad An I/O error occurred reading or writing an extent allocation unit summary. The VX_FULLFSCK flag is set. If the VX_FULLFSCK flag can t be set, the file system is disabled. Check the console log for I/O errors. If the problem is a disk failure, replace the disk. If the problem is not related to an I/O failure, find out how the disk became corrupted. If no user or process was writing to the device, report the problem to your customer support organization. Unmount the file system and use fsck to run a full structural check. 82 Administrators Guide
Code: 058 Code: 058 Code: 059 Code: 060 WARNING: msgcnt x: vxfs: mesg 058: vx_isum_bad - mount_point file system inode allocation unit summary number number marked bad An I/O error occurred reading or writing an inode allocation unit summary. The VX_FULLFSCK flag is set. If the VX_FULLFSCK flag can t be set, the file system is disabled. Check the console log for I/O errors. If the problem is a disk failure, replace the disk. If the problem is not related to an I/O failure, find out how the disk became corrupted. If no user or process was writing to the device, report the problem to your customer support organization. Unmount the file system and use fsck to run a full structural check. WARNING: msgcnt x: vxfs: mesg 059: vx_snap_getbitbp - mount_point snapshot file system bitmap write error An I/O error occurred while writing to the snapshot file system bitmap. There is no problem with the snapped file system, but the snapshot file system is disabled. Check the console log for I/O errors. If the problem is a disk failure, replace the disk. If the problem is not related to an I/O failure, find out how the disk became corrupted. If no user or process was writing to the device, report the problem to your customer support organization. Restart the snapshot on an error free disk partition. Rerun any backups that failed when the error occurred. WARNING: msgcnt x: vxfs: mesg 060: vx_snap_getbitbp - mount_point snapshot file system bitmap read error An I/O error occurred while reading the snapshot file system bitmap. There is no problem with snapped file system, but the snapshot file system is disabled. Appendix C, Supported VxFS Error Conditions 83
Code: 061 Code: 061 Code: 062 Code: 063 Check the console log for I/O errors. If the problem is a disk failure, replace the disk. If the problem is not related to an I/O failure, find out how the disk became corrupted. If no user or process was writing to the device, report the problem to your customer support organization. Restart the snapshot on an error free disk partition. Rerun any backups that failed when the error occurred. WARNING: msgcnt x: vxfs: mesg 061: vx_resize - mount_point file system remount failed During a file system resize, the remount to the new size failed. The VX_FULLFSCK flag is set and the file system is disabled. Unmount the file system and use fsck to run a full structural check. After the check, the file system shows the new size. NOTICE: msgcnt x: vxfs: mesg 062: vx_attr_creatop invalid disposition returned by attribute driver A registered extended attribute intervention routine returned an invalid return code to the VxFS driver during extended attribute inheritance. Determine which vendor supplied the registered extended attribute intervention routine and contact their customer support organization. WARNING: msgcnt x: vxfs: mesg 063: vx_fset_markbad - mount_point file system mount_point fileset (index number) marked bad An error occurred while reading or writing a fileset structure. VX_FULLFSCK flag is set. If the VX_FULLFSCK flag can t be set, the file system is disabled. Unmount the file system and use fsck to run a full structural check. 84 Administrators Guide
Code: 064 Code: 064 Code: 066 Code: 067 WARNING: msgcnt x: vxfs: mesg 064: vx_ivalidate - mount_point file system inode number version number exceeds fileset's During inode validation, a discrepancy was found between the inode version number and the fileset version number. The inode may be marked bad, or the fileset version number may be changed, depending on the ratio of the mismatched version numbers. VX_FULLFSCK flag is set. If the VX_FULLFSCK flag can t be set, the file system is disabled. Check the console log for I/O errors. If the problem is a disk failure, replace the disk. If the problem is not related to an I/O failure, find out how the disk became corrupted. If no user or process is writing to the device, report the problem to your customer support organization. In either case, unmount the file system and use fsck to run a full structural check. NOTICE: msgcnt x: vxfs: mesg 066: DMAPI mount event - buffer An HSM (Hierarchical Storage Management) agent responded to a DMAPI mount event and returned a message in buffer. Consult the HSM product documentation for the appropriate response to the message. WARNING: msgcnt x: vxfs: mesg 067: mount of device_path requires HSM agent The file system mount failed because the file system was marked as being under the management of an HSM agent, and no HSM agent was found during the mount. Restart the HSM agent and try to mount the file system again. Appendix C, Supported VxFS Error Conditions 85
Code: 068 Code: 068 Code: 069 Code: 070 WARNING: msgcnt x: vxfs: mesg 068: ncsize parameter is greater than 80% of the vxfs_ninode parameter; increasing the value of vxfs:vxfs_ninode The value auto-tuned for the vxfs_ninode parameter is less than 125% of the ncsize parameter. This message occurs only if one of the system tuneable parameters ncsize, vxfs_ninode, maxusers,or max_nprocs is set manually in the file /etc/system. To prevent this message from occurring, set vxfs_ninode to at least 125% of the value of ncsize. The best way to do this is to adjust ncsize down, rather than adjusting vxfs_ninode up. See Chapter 5, Performance and Tuning, for more information. WARNING: msgcnt x: vxfs: mesg 069: memory usage specified by the vxfs:vxfs_ninode and vxfs:vx_bc_bufhwm parameters exceeds available memory; the system may hang under heavy load The value of the system tuneable parameters vxfs_ninode and vx_bc_bufhwm add up to a value that is more than 66% of the kernel virtual address space or more than 50% of the physical system memory. VxFS inodes require approximately one kilobyte each, so both values can be treated as if they are in units of one kilobyte. To avoid a system hang, reduce the value of one or both parameters to less than 50% of physical memory or to 66% of kernel virtual memory. See Chapter 5, Performance and Tuning, for more information. WARNING: msgcnt x: vxfs: mesg 070: checkpoint checkpoint_name removed from file system mount_point The file system ran out of space while updating a checkpoint. The checkpoint was removed to allow the operation to complete. 86 Administrators Guide
Code: 071 Code: 071 Increase the size of the file system. If the file system size cannot be increased, remove files to create sufficient space for new checkpoints. Monitor capacity of the file system closely to ensure it does not run out of space. See the fsadm_vxfs(1m) man page more information. NOTICE: msgcnt x: vxfs: mesg 071: cleared data I/O error flag in mount_point file system The user data I/O error flag was reset when the file system was mounted. This message indicates e that a read or write error occurred (see Message 038) while the file system was previously mounted. Informational only, no action required Appendix C, Supported VxFS Error Conditions 87
Code: 071 88 Administrators Guide
Supported VCS Error Conditions D VERITAS SNMP Extension natively supports all standard VCS error codes and many out-of-box capacity and rules templates to for health monitoring and event detection. This document contains the list of error codes and provides a brief description for each. For details regarding the SNMP Extension package, please refer to the VERITAS SNMP Extension for service description document. 89
Supported VCS Critical Events Supported VCS Critical Events The following types of events are supported: Critical events: VCS internal errors that require customer to contact Customer Support Events indicating errors and exceptions Warnings Any abnormal events that customers might want to track Supported VCS Critical Daemon Process Error Conditions The availability of critical daemon processes: had, hashadow etc., or any critical process, which needs to present at all, times. On Windows NT and 2000, the SNMP Extension tracks the VERITAS High Availability Engine, cmdserver etc. The SNMP Extension checks for and triggers an event if a VCS process, (ex: had, hashadow) is hung or runaway based on statistical threshold data like CPU percentage and I/O. Supported VCS Standard Error Codes Code: 1000 Code: 1005 This action removes system_name from group. Group must offline on that node before removing. The user tried to dynamically modify the system list of a group using the command hagrp -modify group_name systemlist system_name priority (plus other arguments).the command fails if the group is not first taken offline on the system. Nodename system_name not defined: not modified in SystemList. 90 Administrators Guide
Code: 1010 Code: 1010 Code: 1015 Code: 1020 Code: 2000 The user tried to dynamically add a system to the system list of a group, using the command hagrp -modify group_name SystemList -add system_name priority. Systems that have not been defined in the configuration language or by the command hasys -add system_name cannot be added to the list. Group not offline on node system_name. The user tried to dynamically delete all systems from the SystemList of a group using the command hagrp -modify group_name SystemList -deletekeys. The system won t delete nodes on which the group is not completely offline. Group group_name must be completely offline before removal from SystemList. The user attempted to dynamically delete nodes from the SystemList of a group using the command hagrp -modify group_name SystemList delete system_name (plus other arguments). The system won t delete nodes from the list on which the group is not completely offline. Cannot online a resource on system system_name which is going down. The command hares -online resource_name -sys system_name won t bring the given resource online on the specified system if the VCS server on the system is in the process of shutting down. Appendix D, Supported VCS Error Conditions 91
Code 2001 Code 2001 Code: 2002 Code: 2003 Code: 2004 hacf -dump failed on system_name. The command hacf was unable to dump the configuration to the configuration file on a system specified by system_name. This message will appear in the logs of all systems in the cluster. Cluster::init() unable to add attribute_name attribute. Return return_number mess: message_string This is an internal error of the HAD process. Invalid channel for cluster modify. The message to modify the cluster object was sent from an invalid channel. No username or password in login message. There was no username or password in the message to the engine. Provide a username and associated password in the login to the cluster. User username does not exist. 92 Administrators Guide
Code: 2005 Code: 2005 Code: 2006 Code: 2007 Code: 2008 A request to log in to the cluster as user username failed. The user name does not exist. Provide a correct username or configure the username with the hauser command. User username has wrong password. The password for user username is invalid. Reenter the correct password. User username has no password. There is no password for username. Configure a password for username. UserNames attribute contains no username to check. Security checking not performed. There is no user configured for the cluster. Configure a user for the cluster. Appendix D, Supported VCS Error Conditions 93
Code: 2009 Code: 2009 Code: 2010 Code: 2011 Code: 2012 Local build from disk failed with message: message. Loading the configuration from disk failed for the reason listed in message. Dump of configuration to disk failed with message: message Dumping the configuration to disk failed for the reason listed in message. Node node_id cannot leave cluster. Dumping the configuration to disk failed for node_id. This system cannot leave the cluster until the running configuration is dumped to disk; otherwise, a running configuration can be lost. This message appears with message #2009. All nodes have configuration files marked stale. Unable to form cluster. All systems in the cluster have configurations that were not shutdown gracefully. Start a system with a current configuration, or force a system to build from local disk. See the hasys -force command in Chapter 5 for details. Last system in RUNNING state has failed or the build from the local disk failed. Unable to form a cluster. 94 Administrators Guide
Code: 2013 Code: 2013 Code: 2014 Code: 2015 The configuration on the disk was invalid, or a system that had a RUNNING configuration left the cluster. Try building from a different configuration file or forcing a different system to build from local disk. Cannot find message;no load analysis for this system. The message utility could not be found. No load balancing was performed. Verify that the environment variable VCS_HOME_ENV is set correctly and that the file indicated exists and is executeable. Cannot find message;no link analysis for this system. The message utility could not be found. No link monitoring was performed. Verify that the environment variable VCS_HOME_ENV is set correctly and that the file indicated exists and is executeable. haconfig does not exist! This is a fatal error. The haconfig utility could not be found. This system is unable to load the configuration from disk, and is also unable to dump the running configuration to disk. Appendix D, Supported VCS Error Conditions 95
Code: 2016 Code: 2016 Code: 2017 Code: 2018 Code: 2019 Verify that the environment variable VCS_HOME_ENV is set correctly and that the file indicated exists and is executable. System::set_attribute unable to update attribute. This is an internal error of the had process. System::init() unable to attribute_name attribute. Return return_number mess: message_string This is an internal error of the had process. System::_add_sys():Invalid Channel System::_delete_sys(): Invalid Channel System::_modify()Invalid Channel A change to a system object was attempted. However, the message came from an invalid channel, so the operation was not performed. System::process_system()Unknown major/minor msg = hex message number A change to a system object was attempted. However, the message was invalid for an unknown reason. 96 Administrators Guide
Code: 2020 Code: 2020 Code: 4000 Code: 4001 Code: 4003 Code: 4004 Code: 4005 Error -had not available. The had process is currently not running and is therefore not available for inquiries from the utility. Start the had process. Aborting after syntax errors. Syntax error in the configuration file input. syntax error - name_of_configuration_file Lexical error in configuration file input. Error processing predefined attributes. Unsuccessful open of ipm service. Appendix D, Supported VCS Error Conditions 97
Code: 4006 Code: 4006 Code: 4007 Code: 4008 Code: 4009 Code: 4010 Cannot mkdir name_of_configuration_directory. The command mkdir fails when making the configuration directory. Cannot chdir name_of_configuration_directory. The command chdir fails when changing to the configuration directory. Could not mark configuration stale. lock file errno name_of_file Error opening lock file. Configuration locked. Configuration busy. Retry later. Configuration stale. 98 Administrators Guide
Code: 4011 Code: 4011 Code: 4012 Code: 4013 Code: 4014 Code: 4015 Code: 4016 Cannot create name_of_configuration_file. Error creating configuration file. fdopen name_of_predefined_file fails. Cannot open temporary predefined file. Cannot create.version file. Cannot open name_of_temporary_file. Error opening temporary file. Cannot open name_of_input_file. Error opening input file. Appendix D, Supported VCS Error Conditions 99
Code: 4017 Code: 4017 Code: 4018 Code: 4019 Code: 4020 Code: 4021 Code: 4022 maximum resource nest of level_number exceeded. Configuration file exceeded the maximum level for resource nesting. unknown type name_of_resource_type Unknown resource type in configuration file. Cannot find system system_name. Cannot write to.version file. Scalar attributes cannot have multiple values. Cannot find resource resource_name. Cannot rename existing_main.cf_name to new_main.cf_name. Cannot rename main.cf file. 100 Administrators Guide
Code: 4023 Code: 4023 Cannot open output_file_name. Cannot open output file. Appendix D, Supported VCS Error Conditions 101
Supported VCS Standard Error Conditions for Windows NT/2k platform Supported VCS Standard Error Conditions for Windows NT/2k platform Failed to create directory The agent could not create the Replication Directory on the shared disk. Verify that the shared drive is accessible, and that no file in that drive has the same name as that of the ReplicationDirectory attribute. Insufficient Memory Close applications not is use and check memory usage. Could not add privileges The Registry Replication agent could not add the privileges required to perform registry-related operations. Check the error reported by Windows and contact Technical Support. The name of the local host is invalid Verify that the local host has a valid name. Failed to save the registry The agent could not save the registry. Verify that there is adequate space on the shared disk where the Replication Directory is created. Check the error reported by Windows. Failed to restore the registry This indicates that some applications may have open registry keys. 102 Administrators Guide
The main (root) key is unknown Verify that the specified keys are not open, or set the ForceRestore attribute to 1. The main (root) key is unknown The Keys attribute is incorrectly defined in the configuration file. If the key KLM\\Software\\VERITAS\\ is to be monitored, the first part of the key, HKLM, is the root key. Verify the configuration definition of the Keys attribute. Failed to get the computer name Check the error reported by Windows and contact Technical Support. Duplicate key, or already part of notification At least one key has been defined more than once in the configuration file s Keys attribute. Note that setting up the replication of a key automatically sets up the replication of its subkeys. Verify the configuration definition of the Keys attribute. Delete and Restore not attempted as backup file cannot be created The agent could not create the backup file on the shared disk. Verify that the shared disk has adequate space Open registry keys. The registry synchronization failed because another application may have opened the registry keys. Verify that the registry keys to be replicated are not opened by another application. Appendix D, Supported VCS Error Conditions 103
Agent module and Registry change monitor module are not of the same version. Agent module and Registry change monitor module are not of the same version. The Registry Replication agent consists of two modules: Registry Replication and Registry Monitoring. If the versions of these modules are different, the agent will not come online. Verify the version of both agent modules. Contact Technical Support if problem persists. Could not locate the file filename The agent could not locate the file required for registry replication. Contact Technical Support if problem persists. 104 Administrators Guide
Supported VxVM Error Conditions E VERITAS SNMP Extension natively supports 72 VxVM error codes and many out of box capacity and rules templates to track storage health and error conditions. User can add new rules to check specific volume manager attributes against predefined thresholds. This document contains the list of error codes and provides a brief description for each. For details regarding the SNMP Extension service package, please refer to the VERITAS SNMP Extension service description document. 105
Supported VxVM Capacity and Rules Related Error Conditions Supported VxVM Capacity and Rules Related Error Conditions 1. VxVM license VxFS license about to expire 2. VxVM Disk group free space is critical Disk group free space is below predefined threshold 3. VxVM Disk free space is critical Disk free space is below predefined threshold 4. Read time rule for disk, subdisk, plex and volume. Average read time (ms) is too long for given disk, subdisk, plex and volume. 5. Write time rule for disk, subdisk, plex and volume. Average write time (ms) is too long for given disk, subdisk, plex and volume. 6. Read, write, wait rules for given disk, subdisk, plex and volume. Many other predefined rule templates for disk, subdisk, plex and volume, diskgroup's kernal, capacity and performance attributes. 106 Administrators Guide
Supported VxVM Standard Error Codes Code:1 Code: 2 Code: 3 signal_name [core dumped] vxvm:vxconfigd: ERROR: signal_name [ - core dumped ] Supported VxVM Standard Error Codes The vxconfigd daemon encountered an unexpected signal while starting up. The specific signal is indicated by signal_name. If the signal caused the vxconfigd process to dump core, then that will be indicated. This could be caused by a bug in vxconfigd, particularly if signal_name is Segmentation fault. Alternately, this could have been caused by a user sending vxconfigd a signal with the kill utility. Contact Customer Support. Unrecognized operating mode vxvm:vxconfigd: ERROR: mode_name: Unrecognized operating mode An invalid string was specified as an argument to the -m option. Valid strings are: enable, disable, and boot. Supply a correct option argument. vxconfigd cannot boot-start RAID-5 volumes vxvm:vxconfigd: ERROR: volume_name: vxconfigd cannot boot-start RAID-5 volumes Appendix E, Supported VxVM Error Conditions 107
Code: 4 A volume that vxconfigd should start immediately upon booting thesystem (i.e., the volume for the /usr file system) has a RAID-5 layout. The/usr file system should never be defined on a RAID-5 volume. It is likely that the only recovery for this is to boot the Volume Manager from a network-mounted root file system (or from a CD-ROM), and reconfigure the /usr file system to be defined on a regular non-raid-5 volume. Code: 4 Cannot get all disk groups from the kernel vxvm:vxconfigd: ERROR: Cannot get all disk groups from the kernel: reason This is an internal Volume Manager error. This error should not occur unless there is a bug in the Volume Manager. Contact Customer Support for more information. Code: 5 Cannot get all disks from the kernel vxvm:vxconfigd: ERROR: Cannot get all disks from the kernel: reason This is an internal Volume Manager error. This error should not occur unless there is a bug in the Volume Manager. Contact Customer Support for more information. Code: 6 Cannot get kernel transaction state 108 Administrators Guide
Code: 7 vxvm:vxconfigd: ERROR: Cannot get kernel transaction state: reason This is an internal Volume Manager error. This error should not occur unless there is a bug in the Volume Manager. Contact Customer Support for more information. Code: 7 Cannot get private storage from kernel vxvm:vxconfigd: ERROR: Cannot get private storage from kernel: reason This is an internal Volume Manager error. This error should not occur unless there is a bug in the Volume Manager. Contact Customer Support for more information. Code: 8 Cannot get private storage size from kernel vxvm:vxconfigd: ERROR: Cannot get private storage size from kernel: reason This is an internal Volume Manager error. This error should not occur unless there is a bug in the Volume Manager. Contact Customer Support for more information. Code: 9 Cannot get record from the kernel Appendix E, Supported VxVM Error Conditions 109
Code: 10 Code: 10 Code: 11 vxvm:vxconfigd: ERROR: Cannot get record record_name from the kernel: reason This is an internal Volume Manager error. This error should not occur unless there is a bug in the Volume Manager. Contact Customer Support for more information. Cannot kill existing daemon, pid=process-id vxvm:vxconfigd: ERROR: Cannot kill existing daemon, pid=process-id The -k (kill existing vxconfigd process) option was specified, but a running configuration daemon process could not be killed. A configuration daemon process, for purposes of this discussion, is any process that opens the /dev/vx/config device (only one process can open that device at a time). If there is a configuration daemon process already running, then the -k option causes a SIGKILL signal to be send to that process. If, within a certain period of time, there is still a running configuration daemon process, then the above error message will be displayed. This error can result from a kernel error that has made the configuration daemon process unkillable, from some other kind of kernel error, or from some other user starting another configuration daemon process after the SIGKILL signal. This last condition can be tested for by running vxconfigd -k again. If the error message appears again, contact Customer Support. Cannot make directory vxvm:vxconfigd: ERROR: Cannot make directory directory_path: reason vxconfigd failed to create a directory that it expects to be able to create. Directories that vxconfigd might try to create are: /dev/vx/dsk, /dev/vx/rdsk, and /var/vxvm/tempdb. Also, for each disk group, /dev/vx/dsk/diskgroup and 110 Administrators Guide
Code: 12 Code: 12 Code: 13 /dev/vx/rdsk/diskgroup directories are created. The system error related to the failure is given in reason. A system error of No such file or directory indicates that one of the prefix directories (for example, /var/vxvm) does not exist. This type of error normally implies that the VxVM packages were installed incorrectly. Such an error can also occur if alternate file or directory locations are specified on the command line, using the -x option. The _VXVM_ROOT_DIR environment variable may also relocate to a directory that lacks a var/vxvm subdirectory. Try to create the directory manually and then issue the command vxdctl enable. If the error is due to incorrect installation of the VxVM packages, try to add the VxVM packages again. Cannot open /etc/vfstab vxvm:vxconfigd: ERROR: Cannot open /etc/vfstab: reason vxconfigd could not open the /etc/vfstab file, for the reason given. The /etc/vfstab file is used to determine which volume (if any) to use for the /usr file system. If the /etc/vfstab file cannot be opened, vxconfigd prints the above error message and exits. This error implies that your root file system is currently unusable. You may be able to repair your root file system by mounting the root file system after booting from a network or CD-ROM root file system. If the root file system is defined on a volume, then see the procedures defined for recovering from a failed root file system in the Recovery appendix. Cannot recover operation in progress vxvm:vxconfigd: ERROR: Cannot recover operation in progress Failed to get group group from the kernel: error Appendix E, Supported VxVM Error Conditions 111
Code: 14 Code: 14 Code: 15 This is an internal Volume Manager error. This error should not occur unless there is a bug in the Volume Manager. Contact Customer Support for more information. Cannot reset VxVM kernel vxvm:vxconfigd: ERROR: Cannot reset VxVM kernel: reason The -r reset option was specified to vxconfigd, but the VxVM kernel drivers could not be reset. The most common reason for this is A virtual disk device is open. That error implies that a VxVM tracing device or volume device is open. If, for some reason, you really want to reset the kernel devices, you will need to track down and kill all processes that have a volume or VxVM tracing device open. Also, if any volumes are mounted as file systems, unmount those file systems. An error reason other than A virtual disk device is open should not normally occur unless there is a bug in the operating system or in the Volume Manager. Cannot start volume, no valid complete plexes vxvm:vxconfigd: ERROR: Cannot start volume volume, no valid complete plexes This error indicates that the volume for either the root or /usr file system cannot be started because the volume contains no valid plexes. This can happen, for example, if disk failures have caused all plexes to be unusable. It can also happen as a result of Actions that caused all plexes to become unusable (for example, forcing the dissociation of subdisks or detaching, dissociation, or offlining of plexes). 112 Administrators Guide
Code: 16 Code: 16 Code: 17 It is possible that this error results from a drive that failed to spin up. If so, rebooting may fix the problem. If that does not fix the problem, then the only recourse is to restore the root or /usr file system or to reinstall the system. Restoring the root or /usr file system requires that you have a valid backup. See the Recovery appendix for information on how to fix problems with root or /usr file system volumes. Cannot start volume, no valid plexes vxvm:vxconfigd: ERROR: Cannot start volume volume, no valid plexes This error indicates that the volume for either the root or /usr file system cannot be started because the volume contains no valid plexes. This can happen, for example, if disk failures have caused all plexes to be unusable. It can also happen as a result of Actions that caused all plexes to become unusable (for example, forcing the dissociation of subdisks or detaching, dissociating, or offlining plexes). It is possible that this error results from a drive that failed to spin up. If so, rebooting may fix the problem. If that does not fix the problem, then the only recourse is to restore the root or /usr file system or to reinstall the system. Restoring the root or /usr file system requires that you have a valid backup. See the Recovery appendix for information on how to fix problems with root or /usr file system volumes. Cannot start volume, volume state is invalid vxvm:vxconfigd: ERROR: Cannot start volume volume, volume state is invalid The volume for the root or /usr file system is in an unexpected state (not ACTIVE, CLEAN, SYNC or NEEDSYNC). This should not happen unless the system administrator circumvents the mechanisms used by the Volume Manager to create these volumes. The only recourse is to bring up the Volume Manager on a CD-ROM or NFS-mounted root file system and to fix the state of the volume. See the Recovery appendix for further information. Appendix E, Supported VxVM Error Conditions 113
Code: 18 Code: 18 Code: 19 Code: 20 Cannot store private storage into the kernel vxvm:vxconfigd: ERROR: Cannot store private storage into the kernel: error This is an internal Volume Manager error. This error should not occur unless there is a bug in the Volume Manager. Contact Customer Support for more information. Differing version of vxconfigd installed vxvm:vxconfigd: ERROR: Differing version of vxconfigd installed A vxconfigd daemon was started after the stopping of an earlier vxconfigd with a non-matching version number. This can happen, for example, if you upgrade from an earlier release of VxVM to VxVM 2.3 and run vxconfigd without a reboot. To fix, reboot the system. Disk, group, device: not updated with new host ID vxvm:vxconfigd: ERROR: Disk disk, group group, device device: not updated with new host ID Error: reason This can result from using vxdctl hostid to change the VxVM host ID for the system. The error indicates that one of the disks in a disk group could not be updated with the new host ID. Most likely, this indicates that the given disk has become inaccessible or has failed in some other way. 114 Administrators Guide
Code: 21 Code: 21 Try running the following to determine whether the disk is still operational: vxdisk check device If the disk is no longer operational, vxdisk should print a message such as: device: Error: Disk write failure This will result in the disk being taken out of active use in its disk group, if it has not been taken out of use already. If the disk is still operational (which should not be the case), vxdisk will print: device: Okay If the disk is listed as Okay, try vxdctl hostid again. If it still results in an error, contact Customer Support. Disk group, Disk: Cannot auto-import group vxvm:vxconfigd: ERROR: Disk group group, Disk disk: Cannot auto-import group: reason On system startup, vxconfigd failed to import the disk group associated with the named disk. A message related to the specific failure is given in reason. Additional error messages may be displayed that give more information on the specific error. In particular, this is often followed by: vxvm:vxconfigd: ERROR: Disk group group: Errors in some configuration copies: Disk device, copy number: Block bno: error (plus other arguments). The most common reason for auto-import failures is excessive numbers of disk failures, making it impossible for the Volume Manager to find correct copies of the disk group configuration database and kernel update log. Disk groups usually have enough copies of this configuration information to make such import failures unlikely. A more serious failure is indicated by error types of: Format error in configuration copy Invalid magic number Invalid block number Duplicate record in configuration Configuration records are inconsistent Appendix E, Supported VxVM Error Conditions 115
Code: 23 Code: 23 These errors indicate that all configuration copies have become corrupt (due to disk failures, writing on the disk by an application or the administrator, or bugs in the Volume Manager). Some correctable errors may be indicated by other error messages that appear in conjunction with the auto-import failure message. Look up those other errors for more information on their cause. Failure of an auto-import implies that the volumes in that disk group will not be available for use. If there are file systems on those volumes, then the system may yield further errors resulting from inability to access the volume when mounting the file system. If the error is clearly caused by excessive disk failures, then you may have to recreate the disk group and restore contents of any volumes from a backup. There may be other error messages that appear which provide further information. See those other error messages for more information on how to proceed. If those errors do not make it clear how to proceed, contact Customer Support. CODE:22 Disk group, Disk: Group name collides with record in rootdg vxvm:vxconfigd: ERROR: Disk group group, Disk device: Group name collides with record in rootdg The name of a disk group that is being imported conflicts with the name of a record in the rootdg disk group. VxVM does not allow this kind of conflict because of the way the /dev/vx/dsk directory is organized: devices corresponding to records in the root disk group share this directory with subdirectories for each disk group. Either remove or rename the conflicting record in the root disk group, or rename the disk group on import. See the vxdg(1m) manual page for information on how to use the import operation to rename a disk group. Disk group, Disk: Skip disk group with duplicate name vxvm:vxconfigd: ERROR: Disk group group, Disk device: Skip disk group with duplicate name 116 Administrators Guide
Code: 23 Two disk groups with the same name are tagged for auto-importing by the same host. Disk groups are identified both by a simple name and by a long unique identifier (disk group ID) assigned when the disk group is created. Thus, this error indicates that two disks indicate the same disk group name but a different disk group ID. The Volume Manager does not allow you to create a disk group or import a disk group from another machine, if that would cause a collision with a disk group that is already imported. Therefore, this error is unlikely to occur under normal use. However, this error can occur in the following two cases: A disk group cannot be auto-imported due to some temporary failure. If you create a new disk group with the same name as the failed disk group and reboot, then the new disk group will be imported first, and the auto-import of the older disk group will fail with group with duplicate name (more recently modified disk groups have precedence over older disk groups). A disk group is deported from one host using the -h option to cause the disk group to be auto-imported on reboot from another host. If the second host was already auto-importing a disk group with the same name, then reboot of that host will yield this error. If you want to import both disk groups, then rename the second disk group on import. See the vxdg(1m) manual page for information on how to use the import operation to rename a disk group. Disk group: Cannot recover temp database vxvm:vxconfigd: ERROR: Disk group group: Cannot recover temp database: reason Consider use of "vxconfigd -x cleartempdir" [see vxconfigd(1m)]. This can happen if you kill and restart vxconfigd or you if you disable and enable it with vxdctl disable and vxdctl enable. This error indicates a failure related to reading the file /var/vxvm/tempdb/groupname. This is a temporary file used to store information that is used when recovering the state of an earlier vxconfigd. The file is recreated on a reboot, so this error should never survive a reboot. If you can reboot, do so. If you do not want to reboot, then do the following: Appendix E, Supported VxVM Error Conditions 117
Code: 24 Code: 24 1. Ensure that no vxvol, vxplex, or vxsd processes are running. Use ps -e to search for such processes, and use kill to kill any that you find. You may have to run kill twice to make these processes go away. Killing utilities in this way may make it difficult to make administrative changes to some volumes until the system is rebooted. 2. Run the command: vxconfigd -x cleartempdir 2> /dev/console This will recreate the temporary database files for all imported disk groups. The vxvol, vxplex, and vxsd commands make use of these tempdb files to communicate locking information. If the file is cleared, then locking information can be lost. Without this locking information, two utilities can end up making incompatible changes to the configuration of a volume. Disk group: Disabled by errors vxvm:vxconfigd: ERROR: Disk group group: Disabled by errors This message indicates that some error condition has made it impossible for VxVM to continue to manage changes to a disk group. The major reason for this is that too many disks have failed, making it impossible for vxconfigd to continue to update configuration copies. There should be a preceding error message that indicates the specific error that was encountered. If the disk group that was disabled is the rootdg disk group, then the following additional error should be displayed: vxvm:vxconfigd: ERROR: All transactions are disabled This additional message indicates that vxconfigd has entered the disabled state, which makes it impossible to change the configuration of any disk group, not just rootdg. If the underlying error resulted from a transient failure, such as a disk cabling error, then you may be able to repair the situation by rebooting. Otherwise, the disk group may have to be recreated and restored from a backup. Failure of the rootdg disk group may require reinstallation of the system if your system uses a root or /usr file system defined on a volume. 118 Administrators Guide
Code: 25 Code: 25 Code: 26 Code: 27 Disk group: Errors in some configuration copies: Disk, copy vxvm:vxconfigd: ERROR: Disk group group: Errors in some configuration copies: Disk disk, copy number: [Block number]: reason (plus other arguments). During a failed disk group import, some of the configuration copies in the named disk group were found to have format or other types of errors which make those copies unusable. This message lists all configuration copies that have uncorrected errors, including any appropriate logical block number. If no other reasons are displayed, then this may be the cause of the disk group import failure. If some of the copies failed due to transient errors (such as cable failures), then a reboot or reimport may succeed in importing the disk group. Otherwise, the disk group may have to be recreated from scratch. Disk group: Reimport of disk group failed vxvm:vxconfigd: ERROR: Disk group group: Reimport of disk group failed: reason After vxconfigd was stopped and restarted (or disabled and then enabled), the Volume Manager failed to recreate the import of the indicated disk group. The reason for failure is specified. Additional error messages may be displayed that give further information describing the problem. A major cause for this kind of failure is disk failures that were not addressed before vxconfigd was stopped or disabled. If the problem is a transient disk failure, then rebooting may take care of the condition. Disk group: update failed Appendix E, Supported VxVM Error Conditions 119
Code: 28 Code: 28 Code: 29 vxvm:vxconfigd: ERROR: Disk group group: update failed: reason I/O failures have prevented vxconfigd from updating any active copies of the disk group configuration. This usually implies a large number of disk failures. This error will usually be followed by the error: vxvm:vxconfigd: ERROR: Disk group group: Disabled by errors If the underlying error resulted from a transient failure, such as a disk cabling error, then you may be able to repair the situation by rebooting. Otherwise, the disk group may have to be recreated and restored from a backup. Failed to store commit status list into kernel vxvm:vxconfigd: ERROR: Failed to store commit status list into kernel: reason This is an internal Volume Manager error. This error should not occur unless there is a bug in the Volume Manager. Contact Customer Support for more information. GET_VOLINFO ioctl failed vxvm:vxconfigd: ERROR: GET_VOLINFO ioctl failed: reason This is an internal Volume Manager error. This error should not occur unless there is a bug in the Volume Manager. Contact Customer Support for more information. 120 Administrators Guide
Code: 30 Code: 30 Code: 31 Code: 32 Get of current rootdg failed vxvm:vxconfigd: ERROR: Get of current rootdg failed: reason This is an internal Volume Manager error. This error should not occur unless there is a bug in the Volume Manager. Contact Customer Support for more information. Memory allocation failure vxvm:vxconfigd: ERROR: Memory allocation failure This implies that there is insufficient memory to start up the Volume Manager and to get the volumes for the root and /usr file systems running. This error should not normally occur, unless your system has very small amounts of memory. Adding just swap space will probably not help because this error is most likely to occur early in the boot sequence, before swap areas have been added. Mount point: volume not in rootdg disk group vxvm:vxconfigd: ERROR: Mount point path: volume not in rootdg disk group The volume device listed in the /etc/vfstab file for the given mount-point directory (normally /usr) is listed as in a disk group other than rootdg. This error should not occur if the standard VxVM procedures are used for encapsulating the disk containing the /usr file system. Appendix E, Supported VxVM Error Conditions 121
Code: 33 Code: 33 Code: 34 You will need to boot the Volume Manager from a network or CD-ROM mounted root file system. Then, start up the Volume Manager using fixmountroot on a valid mirror disk of the root file system. After starting VxVM, mount the root file system volume and edit the /etc/vfstab file. Change the file to use a direct partition for the file system. There should be a comment in the /etc/vfstab file that indicates which partition to use, for example: #NOTE: volume usr (/usr) encapsulated partition c0t3d0s5 No convergence between root disk group and disk list vxvm:vxconfigd: ERROR: No convergence between root disk group and disk list Disks in one version of rootdg: device type=device_type info=devinfo (plus other arguments). Disks in alternate version of rootdg: device type=device_type info=devinfo (plus other arguments). This message can appear when vxconfigd is not running in autoconfigure mode (see the vxconfigd(1m) manual page) and when, after several retries, it can not resolve the set of disks belonging to the root disk group. The algorithm for non-autoconfigure disks is to scan disks listed in the /etc/vx/volboot file and then examine the disks to find a database copy for the rootdg disk group. The database copy is then read to find the list of disk access records for disks contained in the group. These disks are then examined to ensure that they contain the same database copy. As such, this algorithm expects to gain convergence on the set of disks and the database copies contained on them. If a loop is entered and convergence cannot be reached, then this message will appear and the root disk group importation will fail. Reorganizing the physical locations of the devices attached to the system may break the deadlock. Failing this, contact Customer Support. Open of directory failed vxvm:vxconfigd: ERROR: Open of directory directory failed: reason 122 Administrators Guide
Code: 35 Code: 35 Code: 36 An open failed for the /dev/vx/dsk or /dev/vx/rdsk directory (or a subdirectory of either of those directories). The only likely cause of such a failure should be that the directory was removed by the administrator or by an errant program. For this case, the reason should be No such file or directory. An alternate possible cause is an I/O failure. If the error was No such file or directory, then create the directory (using mkdir). Then run the command vxdctl enable. If the error was an I/O error, then there may be other serious damage to the root file system. You may need to reformat your root disk and restore the root file system from backup. Contact your system vendor or consult your system documentation. Read of directory failed vxvm:vxconfigd: ERROR: Read of directory directory failed: reason There was a failure in reading the /dev/vx/dsk or /dev/vx/rdsk directory (or a subdirectory of either of those directories). The only likely cause of this error is an I/O failure on the root file system. If the error was an I/O error, then there may be other serious damage to the root file system. You may need to reformat your root disk and restore the root file system from backup. Contact your system vendor or consult your system documentation. System boot disk does not have a valid root plex vxvm:vxconfigd: ERROR: System boot disk does not have a valid root plex Please boot from one of the following disks: Disk: diskname Device: device (plus other arguments). The system is configured to use a volume for the root file system, but was not booted on a disk containing a valid mirror of the root volume. Disks containing valid root mirrors are listed as part of the error message. A disk is usable as a boot disk if there is a root mirror on that disk which is not stale or offline. Appendix E, Supported VxVM Error Conditions 123
Code: 37 Code: 37 Code: 38 Try to boot from one of the disks named in the error message. Under Solaris, you may be able to boot using a device alias for one of the named disks. For example, try: boot vx-diskname System startup failed vxvm:vxconfigd: ERROR: System startup failed Either the root or the /usr file system volume could not be started, rendering the system unusable. The error that resulted in this condition should appear prior to this error message. Look up other error messages appearing on the console and take the actions suggested in the descriptions of those messages. There is no volume configured for the root device vxvm:vxconfigd: ERROR: There is no volume configured for the root device The system is configured to boot from a root file system defined on a volume, but there is no root volume listed in the configuration of the rootdg disk group. There are two possible causes of this error: Case 1: The /etc/system file was erroneously updated to indicate that the root device is /pseudo/vxio@0:0. This should happen only as a result of direct manipulation by the administrator. Case 2: The system somehow has a duplicate rootdg disk group, one of which contains a root file system volume and one of which does not, and vxconfigd somehow chose the wrong one. Since vxconfigd chooses the more recently accessed version of rootdg, this error can happen if the system clock was updated incorrectly at some point (causing the 124 Administrators Guide
Code: 39 Code: 39 Code: 40 apparent access order of the two disk groups to be reversed). This can also happen if some disk group was deported and renamed to rootdg with locks given to this host. In case 1, boot the system on a CD-ROM or networking-mounted root file system, directly mount the disk partition of the root file system, and remove the following lines from /etc/system: rootdev:/pseudo/vxio@0:0 set vxio:vol_rootdev_is_volume=1 In case 2, either boot with all drives in the offending version of rootdg turned off, or import and rename [see vxdg(1m)] the offending rootdg disk group from another host. In the case of turning off drives, run the following command after booting: vxdg flush rootdg This will update time stamps on the imported version of rootdg, which should make the correct version appear to be the more recently accessed. If this does not correct the problem, then contact Customer Support. Unexpected configuration tid for group found in kernel vxvm:vxconfigd: ERROR: Unexpected configuration tid for group group found in kernel This is an internal Volume Manager error. This error should not occur unless there is a bug in the Volume Manager. Contact Customer Support for more information. Unexpected error during volume reconfiguration vxvm:vxconfigd: ERROR: Unexpected error during volume volume reconfiguration: reason Appendix E, Supported VxVM Error Conditions 125
Code: 41 Code: 41 Code: 42 Code: 43 This is an internal Volume Manager error. This error should not occur unless there is a bug in the Volume Manager. Contact Customer Support for more information. Unexpected error fetching disk for volume vxvm:vxconfigd: ERROR: Unexpected error fetching disk for disk volume: reason This is an internal Volume Manager error. This error should not occur unless there is a bug in the Volume Manager. Contact Customer Support for more information. Unexpected values stored in the kernel vxvm:vxconfigd: ERROR: Unexpected values stored in the kernel This is an internal Volume Manager error. This error should not occur unless there is a bug in the Volume Manager. Contact Customer Support for more information. Version number of kernel does not match vxconfigd vxvm:vxconfigd: ERROR: Version number of kernel does not match vxconfigd 126 Administrators Guide
Code: 44 Code: 44 The release of vxconfigd does not match the release of the Volume Manager kernel drivers. This should happen only as a result of upgrading VxVM, and then running vxconfigd without a reboot. Reboot the system. If that does not cure the problem, then add the VxVM packages again. Volume for mount point /usr not found in rootdg disk group vxvm:vxconfigd: ERROR: Volume volume for mount point /usr not found in rootdg disk group The system is configured to boot with /usr mounted on a volume, but the volume associated with /usr is not listed in the configuration of the rootdg disk group. There are a couple of possible causes of this error: The /etc/vfstab file was erroneously updated to indicate the devicefor the /usr file system is a volume, but the volume named is not in therootdg disk group. This should happen only as a result of directmanipulation by the administrator. The system somehow has a duplicate rootdg disk group, one of which contains the /usr file system volume and one of which does not (or usesa different volume name), and vxconfigd somehow chose the wrong rootdg. Since vxconfigd chooses the more recently accessed version of rootdg, this error can happen if the system clock was updated incorrectly at some point (causing the apparent access order of the two disk groups to be reversed). This can also happen if some disk group was deported and renamed to rootdg with locks given to this host. In case 1, boot the system on a CD-ROM or networking-mounted root file system. If the root file system is defined on a volume, then start and mount the root volume using the procedures defined in the Recovery appendix. If the root file system is not defined on a volume, then just mount the root file system directly. Edit the /etc/vfstab file to correct the entry for the /usr file system. In case 2, either boot with all drives in the offending version of rootdg turned off, or import and rename [see vxdg(1m)] the offending rootdg disk group from another host. In the case of turning off drives, run the following command after booting: vxdg flush rootdg Appendix E, Supported VxVM Error Conditions 127
Code: 45 Code: 45 Code: 46 This will update time stamps on the imported version of rootdg, which should make the correct version appear to be the more recently accessed. If this does not correct the problem, then contact Customer Support. cannot open /dev/vx/config vxvm:vxconfigd: ERROR: cannot open /dev/vx/config: reason The /dev/vx/config device could not be opened. vxconfigd uses this device to communicate with the Volume Manager kernel drivers. The reason string indicates the reason for the open failure. The most likely reason is Device is already open. This reason indicates that some process (most likely vxconfigd) already has /dev/vx/config open. Other less likely reasons are No such file or directory or No such device or address. For either of these two reasons, the two likely causes are: The Volume Manager package installation did not complete correctly. The device node was removed by the administrator or by an errant shell script. For the reason Device is already open, if you really want to run vxconfigd, then stop or kill the old one. You can kill whatever process has vxconfigd open by running the command: vxdctl -k stop For other failure reasons, consider re-adding the base Volume Manager package. This will reconfigure the device node and re-install the Volume Manager kernel device drivers. See the VERITAS Volume Manager Installation Guide for information on how to add the package using pkgadd. If you cannot re-add the package, then contact Customer Support for more information. enable failed vxvm:vxconfigd: ERROR: enable failed: reason 128 Administrators Guide
Code: 47 Code: 47 Regular startup of vxconfigd failed for the stated reason. This error can also result from the command vxdctl enable. This error may include the following additional text: additional-reason; aborting This message indicates that the failure was fatal and that vxconfigd is forced to exit. The most likely cause that results in an abort is inability to create IPC channels for communicating with other utilities. additional-reason; transactions are disabled This message indicates that vxconfigd is continuing to run, but no configuration updates are possible until the error condition is repaired. Additionally, this may be followed with: vxvm:vxconfigd: ERROR: Disk group group: Errors in some configuration copies: Disk device, copy number: Block bno: error (plus other arguments). Reasons for failure vary considerably. Other error messages may be displayed that further indicate the underlying problem. If the Errors in some configuration copies error occurs, then that may indicate the problem. Evaluate other error messages occurring with this one to determine the root cause of the problem. Make changes suggested by the other errors and then retry the command. /dev/vx/info vxvm:vxconfigd: ERROR: /dev/vx/info: reason The /dev/vx/info device could not be opened, or did not respond to a Volume Manager kernel request. This error most likely indicates one of the following: The Volume Manager package installation did not complete correctly. The device node was removed by the administrator or by an errant shell script. Consider re-adding the base Volume Manager package. This will reconfigure the device node and re-install the Volume Manager kernel device drivers. See the VERITAS Volume Manager Installation Guide for information on how to add the package using pkgadd. Appendix E, Supported VxVM Error Conditions 129
VM Fatal Error Messages VM Fatal Error Messages Code: 48 Code: 49 Code: 50 Disk group rootdg: Inconsistency Not loaded into kernel vxvm:vxconfigd: FATAL ERROR: Disk group rootdg: Inconsistency -- Not loaded into kernel This is an internal Volume Manager error. This error should not occur unless there is a bug in the Volume Manager. Contact Customer Support for more information. Group: Cannot update kernel vxvm:vxconfigd: FATAL ERROR: Group group: Cannot update kernel This is an internal Volume Manager error. This error should not occur unless there is a bug in the Volume Manager. Contact Customer Support for more information. Interprocess communication failure vxvm:vxconfigd: FATAL ERROR: Interprocess communication failure: reason This is an internal Volume Manager error. This error should not occur unless there is a bug in the Volume Manager. 130 Administrators Guide
Code: 51 Code: 51 Code: 52 Code: 53 Contact Customer Support for more information. Invalid status stored in kernel vxvm:vxconfigd: FATAL ERROR: Invalid status stored in kernel This is an internal Volume Manager error. This error should not occur unless there is a bug in the Volume Manager. Contact Customer Support for more information. Memory allocation failure during startup vxvm:vxconfigd: FATAL ERROR: Memory allocation failure during startup This implies that there is insufficient memory to start up the Volume Manager and to get the volumes for the root and /usr file systems running. This error should not normally occur, unless your system has very small amounts of memory. Adding just swap space probably will not help, because this error is most likely to occur early in the boot sequence, before swap areas have been added. Rootdg cannot be imported during boot vxvm:vxconfigd: FATAL ERROR: Rootdg cannot be imported during boot Appendix E, Supported VxVM Error Conditions 131
Code: 54 Code: 54 This is an internal Volume Manager error. This error should not occur unless there is a bug in the Volume Manager. Contact Customer Support for more information. Unexpected threads failure vxvm:vxconfigd: FATAL ERROR: Unexpected threads failure: reason This is an unexpected operating system error. This error should not occur unless there is a bug in the Volume Manager or in the Solaris multi-threading libraries. Contact Customer Support for more information. VM Kernel Warning Messages Code: 106 Read error on mirror of volume vxvm:vxio:warning: read error on mirror plex of volume volume offset offset length length An error was detected while reading a mirror. This error may lead to further action shown by later error messages. If the volume is mirrored, no action is necessary at this time since the alternate mirror s contents will be written to the failing mirror; this is often sufficient to correct media failures. If this error occurs often but never leads to a plex detach, there may be a marginal region on the disk at the position shown. It may eventually be necessary to remove data from this disk (see 132 Administrators Guide
Code: 107 Code: 107 Code: 108 the vxevac(1m) manual page) and then to reformat the drive. In the unmirrored case, this message indicates that some data could not be read. The file system or other application reading the data may report an additional error, but in either event, data has been lost. The volume can be partially salvaged and moved to another location if desired. Write error on mirror of volume offset length vxvm:vxio:warning: write error on mirror plex of volume volume offset offset length length An error was detected while writing a mirror. This error will generally be followed by a detach message, unless the volume is un-mirrored. The disk reporting the error is failing to correctly store written data. If the volume is not mirrored, consider removing the data and reformatting the disk. If the volume is mirrored, it will become detached and you should consider replacing or reformatting the disk. If this error occurs often but never leads to a plex detach, there may be a marginal region on the disk at the position shown. It may eventually be necessary to remove data from this disk (see the vxevac(1m) manual page) and then to reformat the drive. Object detached from volume vxvm:vxio:warning: object plex detached from volume volume An uncorrectable error was detected by the mirroring code and a mirror copy was detached. To restore redundancy, it may be necessary to add another mirror. The disk on which the failure occurred should be evacuated and reformatted, if possible. If the drive has failed completely, it may need to be replaced. Appendix E, Supported VxVM Error Conditions 133
Code: 109 Code: 109 Code: 110 Code: 111 Overlapping mirror detached from volume vxvm:vxio:warning: Overlapping mirror plex detached from volume volume An error has occurred on the last complete plex in the mirrored volume. Any sparse mirrors that map the failing region must be detached so that they cannot be accessed to satisfy that failed region inconsistently. Thismessage indicates that such an overlapping mirror was found and is being detached. No Action is directly necessary. The message indicates that the volume may have left some data inaccessible at the failing region and that it is no longer redundantly stored. Kernel log full vxvm:vxio:warning: Kernel log full: volume detached A plex detach failed because the kernel log was full. As a result, the mirrored volume will become detached. It is unlikely that this condition could ever occur. The only corrective action for the detached volume is to reboot the system. Kernel log update failed vxvm:vxio:warning: Kernel log update failed: volume detached A plex detach failed because the kernel log could not be flushed to disk. As a result, the mirrored volume will become detached. This could be caused by all the disks containing a kernel log going bad. 134 Administrators Guide
Code: 112 Code: 112 Code: 113 Code: 114 Correct the failed disks so that kernel logging can once again function. Detaching RAID-5 volume vxvm:vxio:warning: detaching RAID-5 raidvol Either a double-failure condition in the RAID-5 volume has been detected in the kernel or some other fatal error is preventing further use of the array. If two or more drives were lost due to a controller or power failure, then once the disks can be re-attached to the system, they should be recovered using the vxrecover utility. Check the console for other errors that may provide additional information as to the nature of the failure. Object detached from RAID-5 volume vxvm:vxio:warning: object subdisk detached from RAID-5 raidvol at column column offset offset A subdisk was detached from a RAID-5 volume at the specified column number and offset. This is caused by the failure of a disk or an uncorrectable error occurring on that disk. Check the console for other error messages indicating the cause of the failure. If the disk has failed, then it should be replaced as soon as possible. RAID-5 volume entering degraded mode operation Appendix E, Supported VxVM Error Conditions 135
Code: 115 Code: 115 Code: 118 vxvm:vxio:warning: RAID-5 raidvol entering degraded mode operation This message occurs when an uncorrectable error has forced the detach of a subdisk. At this point, not all data disks exist to provide the data upon request. Instead, parity regions are required to regenerate the data for each stripe in the array. Accesses will consequently take longer and will involve reading from all drives in the stripe. Check the console for other error messages indicating the cause of the failure. If the disk has failed, it should be replaced as soon as possible. Double failure condition detected on RAID-5 volume vxvm:vxio:warning: Double failure condition detected on RAID-5 raidvol Double-failures occur if I/O errors are received at the same altitude in the array from more than one column of the array. This could be caused by a controller failure causing more than a single drive to become unavailable; by the loss of a second drive after having run in a degraded state for significant periods of time; or by two separate disk drives failing simultaneously (which is unlikely to happen). If the condition is correctable and the drives recoverable, the conditions should be corrected. The volume can then be recovered using the vxrecover(1m) command. Overlapping ilocks vxvm:vxio:warning: check_ilocks: overlapping ilocks: offset for length, offset for length This is an internal Volume Manager problem. This warning should not occur unless there is a bug in the Volume Manager. 136 Administrators Guide
Code: 119 Code: 119 Code: 120 Code: 121 Contact Customer Support for more information. Illegal vminor encountered vxvm:vxio: WARNING: Illegal vminor encountered If a volume device other than the root volume device is opened before a configuration has been loaded, this message could result. No action should be necessary; an attempt to access a volume device was made before the volume daemon (vxconfigd) loaded the volume configuration. Under normal startup conditions, this message should not occur. If the operation is necessary, start the Volume Manager and re-attempt the operation. Uncorrectable read error vxvm:vxio:warning: object_type object_name block offset: Uncorrectable read error A read or write operation from the specified object failed. An error will be returned to the application. This error represents lost data. The data may need to be restored and failed media may need to be repaired. Depending on the type of object failing and on the type of recovery suggested for that type, an appropriate recovery operation may be necessary. Uncorrectable read/write error vxvm:vxio:warning: object_type object_name block offset: Appendix E, Supported VxVM Error Conditions 137
Code: 122 Code: 122 Code: 123 Uncorrectable read error on object_type object_name block offset vxvm:vxio:warning: object_type object_name block offset: Uncorrectable write error on object_type object_name block offset A read or write operation from the specified object failed. An error will be returned to the application. Although similar to the previous message, this message is able to supply more specific information about the failing object. This error represents lost data. The data may need to be restored and failed media may need to be repaired. Depending on the type of object failing and on the type of recovery suggested for that type, an appropriate recovery operation may be necessary. Root volumes are not supported on your PROM version vxvm:vxio:warning: Root volumes are not supported on your PROM version. The Volume Manager requires the ability to access the PROMs for your SPARC hardware. If the PROMs are not a recent OpenBoot PROM type, then root volumes will not be usable. If you have set up a root volume, then undo the configuration (by running vxunroot or removing the rootdev line from /etc/system) as soon as possible and contact your hardware vendor for an upgrade to your PROM level. Cannot find device number vxvm:vxio:warning: Cannot find device number for boot_path The supplied boot path was retrieved from the PROMs for your system. It cannot be converted to a valid device number. 138 Administrators Guide
Code: 124 Code: 124 Code: 125 Check your PROM settings for the correct boot string. mod_install returned errno vxvm:vxio:warning: mod_install returned errno A call made to the Solaris mod_install() function to load the vxio driver failed. Check your console for additional messages that may explain why the load failed. Also check the console messages log file for any additional messages that were logged but not displayed on the console. subdisk failed in plex in volume vxvm:vxio:warning: subdisk subdisk failed in plex plex in volume volume The kernel has detected a subdisk failure, which may mean that the underlying disk is failing. Check for obvious problems with the disk (such as a disconnected cable). If hot-relocation is enabled and the disk is failing, the subdisk failure may be taken care of automatically. VM Kernel Panic Messages Code: 126 Object association depth overflow vxvm:vxio:panic: Object association depth overflow Appendix E, Supported VxVM Error Conditions 139
Code: 126 This is an internal Volume Manager problem. This warning should not occur unless there is a bug in the Volume Manager. Contact Customer Support for more information. 140 Administrators Guide
Event IDs F This appendix includes the following: A list of the event types and the scripts that they call An explanation of event IDs in SNMP v2 of VERITAS SNMP Extension An explanation of event IDs in the earlier version 1 of the product 141
Events and Scripts Events and Scripts The table below lists the different event types and indicates which event script is called by each type of event. Events and the Scripts they Call Event Types Scripts Called Notes Process Existance Events Logfile Error Message Events psevent logevent NBU Job Events jobevent error code (1-254) NBU General Events nbuevent error code (300-310) Heart Beat Events Collector Based Events NBU Extended Action Events Volume Manager Events File System Events VCS Events VSM Events Events to VST DB For manually generating testing log event hbevent clevent nbuextevent vxvmevent vxfsevent vcsevent vsmevent ene2etrak testevent 142 Administrators Guide
VERITAS SNMP Event IDs in SNMP v2 VERITAS SNMP Event IDs in SNMP v2 abnormaljobevent TYPE NOTIFICATION-TYPE OID 1.3.6.1.4.1.1035.1.1.251.0.1 FULL PATH iso(1).org(3).dod(6).internet(1).private(4).enterprises(1).veritas(1035).netbackup(1).job(1). VRTSnbuJobEvent(251).jobTrap(0).abnormalJobEvent(1) MODULE VRTS-SNMP-MIB PARENT jobtrap NEXT SIBLING normaljobevent STATUS current OBJECTS 1. jobid 2. jobclass 3. jobclient 4. jobsched 5. jobschedtype 6. jobstu 7. jobvolpool 8. joberrcode (job error code can vary from 1 to 254 ) Appendix F, Event IDs 143
normaljobevent 9. joberrmsg 10. jobmaster 11. joberrexpl 12. joberrreco The NetBackup job failed with error. Detailed job, error and corrective action information can be found from the objects associated with this event. normaljobevent TYPE NOTIFICATION-TYPE OID 1.3.6.1.4.1.1035.1.1.251.0.2 FULL PATH iso(1).org(3).dod(6).internet(1).private(4).enterprises(1).veritas(1035).netbackup(1).job(1). VRTSnbuJobEvent(251).jobTrap(0).normalJobEvent(2) MODULE VRTS-SNMP-MIB PARENT jobtrap PREV SIBLING abnormaljobevent STATUS current OBJECTS 1. jobid 2. jobclass 3. jobclient 144 Administrators Guide
nbuextevent nbuextevent 4. jobsched 5. jobschedtype 6. jobstu 7. jobvolpool 8. joberrcode 9. joberrmsg 10. jobmaster 11. joberrexpl 12. joberrreco The NetBackup job finished successfully. Detailed job information can be found from the objects associated with this event. TYPE NOTIFICATION-TYPE OID 1.3.6.1.4.1.1035.1.101.0.1 FULL PATH iso(1).org(3).dod(6).internet(1).private(4).enterprises(1).veritas(1035).netbackup(1).vrtsn buextevent(101).nbuexttrap(0).nbuextevent(1) MODULE VRTS-SNMP-MIB PARENT nbuexttrap STATUS current Appendix F, Event IDs 145
nbbackupfailed OBJECTS 1. nbuindex 2. nbunames 3. nbucommand 4. nburesult 5. nbuoutput The NetBackup extensiable script execution result notificaiton with given exit status and output. nbbackupfailed TYPE NOTIFICATION-TYPE OID 1.3.6.1.4.1.1035.1.251.0.300 FULL PATH iso(1).org(3).dod(6).internet(1).private(4).enterprises(1).veritas(1035).netbackup(1).vrtsn buevents(251).nbutraps(0).dbbackupfailed(300) MODULE VRTS-SNMP-MIB PARENT nbutraps NEXT SIBLING dbbackupisdisabled STATUS current 146 Administrators Guide
dbbackupisdisabled OBJECTS 1. nbuhost 2. nbuerrcode 3. nbuerrmsg The backup of the NB database failed. Investigate why the NB database backup failed. Usually the All Log Entries Report will give you some idea as to why this failed. dbbackupisdisabled TYPE NOTIFICATION-TYPE OID 1.3.6.1.4.1.1035.1.251.0.301 FULL PATH iso(1).org(3).dod(6).internet(1).private(4).enterprises(1).veritas(1035).netbackup(1).vrtsn buevents(251).nbutraps(0).dbbackupisdisabled(301) MODULE VRTS-SNMP-MIB PARENT nbutraps PREV SIBLING dbbackupfailed NEXT SIBLING freezingmedia Appendix F, Event IDs 147
freezingmedia STATUS current OBJECTS 1. nbuhost 2. nbuerrcode 3. nbuerrmsg freezingmedia Backup of the NetBackup database is currently disabled. No Immediate action is required. Periodically NetBackup will disable the ability to perform a NBU database backup while it is performing house-cleaning activities. Should the NB database backup be disabled for an extended period time, review the All Log Entries report to help determine why. TYPE NOTIFICATION-TYPE OID 1.3.6.1.4.1.1035.1.251.0.302 FULL PATH iso(1).org(3).dod(6).internet(1).private(4).enterprises(1).veritas(1035).netbackup(1).vrtsn buevents(251).nbutraps(0).freezingmedia(302) MODULE VRTS-SNMP-MIB PARENT nbutraps PREV SIBLING dbbackupisdisabled 148 Administrators Guide
suspendingmedia NEXT SIBLING suspendingmedia STATUS current OBJECTS 1. nbuhost 2. nbuerrcode 3. nbuerrmsg NetBackup has issued a 'freezing media' message. A piece of media (tape or optical) has been frozen either manually by an administrator or automatically by NetBackup. NetBackup will freeze media that has been previously written in another tape format (tar, ANSI) or that has been written by the NetBackup catalog. NetBackup, by default will not overwrite these tapes unless a bp.conf entry to ALLOW_MEDIA_OVERWRITE is added, see the NetBackup Administrators Guide. Also investigate the All log Entries report and bptm logs for more information. suspendingmedia TYPE NOTIFICATION-TYPE OID 1.3.6.1.4.1.1035.1.251.0.303 FULL PATH iso(1).org(3).dod(6).internet(1).private(4).enterprises(1).veritas(1035).netbackup(1).vrtsn buevents(251).nbutraps(0).suspendingmedia(303) MODULE VRTS-SNMP-MIB Appendix F, Event IDs 149
mediarequiredforrestore PARENT nbutraps PREV SIBLING freezingmedia NEXT SIBLING mediarequiredforrestore STATUS current OBJECTS 1. nbuhost 2. nbuerrcode 3. nbuerrmsg NetBackup has issued a 'suspending media' message. NetBackup tried to use the identified media but had problems reading or writing to the tape. Investigate the cause through the All Log Entries report and / or bptm log. mediarequiredforrestore TYPE NOTIFICATION-TYPE OID 1.3.6.1.4.1.1035.1.251.0.304 FULL PATH iso(1).org(3).dod(6).internet(1).private(4).enterprises(1).veritas(1035).netbackup(1).vrtsn buevents(251).nbutraps(0).mediarequiredforrestore(304) 150 Administrators Guide
downeddrive MODULE VRTS-SNMP-MIB PARENT nbutraps PREV SIBLING suspendingmedia NEXT SIBLING downeddrive STATUS current downeddrive OBJECTS 1. nbuhost 2. nbuerrcode 3. nbuerrmsg Media is needed for an NBU restore operation. The event is sent via a user alarm in the action script. The media identified is required for a restore that has been initiated. The media id is also identified in the Device Management Window. Locate the required tape, place it in the library or tape drive and satisfy the tape mount request through the Device Manager window. You may also deny the request through the Device Manager window. TYPE NOTIFICATION-TYPE OID 1.3.6.1.4.1.1035.1.251.0.305 Appendix F, Event IDs 151
exceededcleaningfrequency FULL PATH iso(1).org(3).dod(6).internet(1).private(4).enterprises(1).veritas(1035).netbackup(1).vrtsn buevents(251).nbutraps(0).downeddrive(305) MODULE VRTS-SNMP-MIB PARENT nbutraps PREV SIBLING mediarequiredforrestore NEXT SIBLING exceededcleaningfrequency STATUS current OBJECTS 1. nbuhost 2. nbuerrcode 3. nbuerrmsg The operator or Media Manager has downed a drive. A tape drive is suspected of having problems. Investigate the All Log Entries report, bptm log, Operating System messages log (/var/adm/messages) to determine the cause. NetBackup may down a drive when having trouble reading a tape, if the tape is used in another drive and also has problems then the tape is suspended and the original downed drive is automatically brough up. exceededcleaningfrequency TYPE NOTIFICATION-TYPE 152 Administrators Guide
exceededcleaningfrequency OID 1.3.6.1.4.1.1035.1.251.0.306 FULL PATH iso(1).org(3).dod(6).internet(1).private(4).enterprises(1).veritas(1035).netbackup(1).vrtsn buevents(251).nbutraps(0).exceededcleaningfrequency(306) MODULE VRTS-SNMP-MIB PARENT nbutraps PREV SIBLING downeddrive NEXT SIBLING exceededmaxmounts STATUS current OBJECTS 1. nbuhost 2. nbuerrcode 3. nbuerrmsg A drive has exceeded its cleaning frequency. This should not normally occur with most TLD devices such as with STK Silo's and DLT tape drives, which automatically handle cleanings. This error message only occurs when NetBackup is handling tape cleanings manually. Appendix F, Event IDs 153
exceededmaxmounts exceededmaxmounts TYPE NOTIFICATION-TYPE OID 1.3.6.1.4.1.1035.1.251.0.307 FULL PATH iso(1).org(3).dod(6).internet(1).private(4).enterprises(1).veritas(1035).netbackup(1).vrtsn buevents(251).nbutraps(0).exceededmaxmounts(307) MODULE VRTS-SNMP-MIB PARENT nbutraps PREV SIBLING exceededcleaningfrequency NEXT SIBLING mountrequest STATUS current OBJECTS 1. nbuhost 2. nbuerrcodenbuerrmsg A media has exceeded its maximum number of mounts. 154 Administrators Guide
mountrequest mountrequest When the media was added to NetBackups' Media Manager, a maximum number of mounts was assigned. The media has reached this limit which normally signals that it should be retired and replaced with another piece of media. Data on the tape is still tracked on the tape and expires once the retention period of the image has passed. TYPE NOTIFICATION-TYPE OID 1.3.6.1.4.1.1035.1.251.0.308 FULL PATH iso(1).org(3).dod(6).internet(1).private(4).enterprises(1).veritas(1035).netbackup(1).vrtsn buevents(251).nbutraps(0).mountrequest(308) MODULE VRTS-SNMP-MIB PARENT nbutraps PREV SIBLING exceededmaxmounts NEXT SIBLING nocleaningtape STATUS current OBJECTS 1. nbuhost 2. nbuerrcode 3. nbuerrmsg Appendix F, Event IDs 155
nocleaningtape A mount request is posted to the system log for a non-robotic device. It may be either for the mount itself or because the media needs 'write enabled'. This message will also be displayed in the Device Manager window. Place the tape in the required non-robotic drive and satisfy the request through the Device Manager window. Deny the request through the Device Manager window. Or remove the media from the tape drive, write enable it and return it to the tape drive. This message is normally issued for tape drives under Operator control rather than AVR or Robotic control. nocleaningtape TYPE NOTIFICATION-TYPE OID 1.3.6.1.4.1.1035.1.251.0.309 FULL PATH iso(1).org(3).dod(6).internet(1).private(4).enterprises(1).veritas(1035).netbackup(1).vrtsn buevents(251).nbutraps(0).nocleaningtape(309) MODULE VRTS-SNMP-MIB PARENT nbutraps PREV SIBLING mountrequest NEXT SIBLING zerocleaningsleft STATUS current 156 Administrators Guide
zerocleaningsleft OBJECTS 1. nbuhost 2. nbuerrcode 3. nbuerrmsg There is no cleaning tape or the cleaning tape is used up. This message only occurs when NetBackup is handling tape drive cleanings and should not normally occur with most TLD devices. zerocleaningsleft TYPE NOTIFICATION-TYPE OID 1.3.6.1.4.1.1035.1.251.0.310 FULL PATH iso(1).org(3).dod(6).internet(1).private(4).enterprises(1).veritas(1035).netbackup(1).vrtsn buevents(251).nbutraps(0).zerocleaningsleft(310) MODULE VRTS-SNMP-MIB PARENT nbutraps PREV SIBLING nocleaningtape STATUS current Appendix F, Event IDs 157
errorvxvmevent OBJECTS 1. nbuhost 2. nbuerrcode 3. nbuerrmsg The cleaning tape has no cleanings left. This message only occurs when NetBackup is handling tape drive cleanings and the cleaning tape(s) have been used to their capacity. This should not normally occur with most TLD devices. errorvxvmevent TYPE NOTIFICATION-TYPE OID 1.3.6.1.4.1.1035.2.251.0.1 FULL PATH iso(1).org(3).dod(6).internet(1).private(4).enterprises(1).veritas(1035).vxvm(2).vrtsvxvm Event(251).vxvmTrap(0).errorVxvmEvent(1) MODULE VRTS-SNMP-MIB PARENT vxvmtrap NEXT SIBLING normalvxvmevent STATUS current 158 Administrators Guide
normalvxvmevent OBJECTS 1. vxvmhost 2. vxvmtype 3. vxvmseverity 4. vxvmerrmsg The VXVM error event. Detailed error and corrective action information can be found from the objects associated with this event. normalvxvmevent TYPE NOTIFICATION-TYPE OID 1.3.6.1.4.1.1035.2.251.0.2 FULL PATH iso(1).org(3).dod(6).internet(1).private(4).enterprises(1).veritas(1035).vxvm(2).vrtsvxvm Event(251).vxvmTrap(0).normalVxvmEvent(2) MODULE VRTS-SNMP-MIB PARENT vxvmtrap PREV SIBLING errorvxvmevent STATUS current OBJECTS 1. vxvmhost 2. vxvmtype Appendix F, Event IDs 159
errorvxfsevent 3. vxvmseverity 4. vxvmerrmsg The normal VXVM event. Detailed event information can be found from the objects associated with this event. errorvxfsevent TYPE NOTIFICATION-TYPE OID 1.3.6.1.4.1.1035.3.251.0.1 FULL PATH iso(1).org(3).dod(6).internet(1).private(4).enterprises(1).veritas(1035).vxfs(3).vrtsvxfsev ent(251).vxfstrap(0).errorvxfsevent(1) MODULE VRTS-SNMP-MIB PARENT vxfstrap NEXT SIBLING normalvxfsevent STATUS current OBJECTS 1. vxfshost 2. vxfsmsgcnt 3. vxfsseverity 4. vxfserrcode 5. vxfserrmsg 160 Administrators Guide
normalvxfsevent The VXFS error event. Detailed error and corrective action information can be found from the objects associated with this event. normalvxfsevent TYPE NOTIFICATION-TYPE OID 1.3.6.1.4.1.1035.3.251.0.2 FULL PATH iso(1).org(3).dod(6).internet(1).private(4).enterprises(1).veritas(1035).vxfs(3).vrtsvxfsev ent(251).vxfstrap(0).normalvxfsevent(2) MODULE VRTS-SNMP-MIB PARENT vxfstrap PREV SIBLING errorvxfsevent STATUS current OBJECTS 1. vxfshost 2. vxfsmsgcnt 3. vxfsseverity 4. vxfserrcode 5. vxfserrmsg Appendix F, Event IDs 161
errorvcsevent errorvcsevent The normal VXFS event. Detailed event information can be found from the objects associated with this event. TYPE NOTIFICATION-TYPE OID 1.3.6.1.4.1.1035.7.251.0.1 FULL PATH iso(1).org(3).dod(6).internet(1).private(4).enterprises(1).veritas(1035).vcs(7).vrtsvcseven t(251).vcstrap(0).errorvcsevent(1) MODULE VRTS-SNMP-MIB PARENT vcstrap NEXT SIBLING normalvcsevent STATUS current OBJECTS 1. vcstag 2. vcsdate 3. vcstime 4. vcserrmsg 5. vcserrcode 6. vcshost 7. vcsres 162 Administrators Guide
normalvcsevent 8. vcsagent 9. vcsnode The VCS error event with TAG_A, TAG_B and TAG_C. Detailed error and corrective action information can be found from the objects associated with this event. normalvcsevent TYPE NOTIFICATION-TYPE OID 1.3.6.1.4.1.1035.7.251.0.2 FULL PATH iso(1).org(3).dod(6).internet(1).private(4).enterprises(1).veritas(1035).vcs(7).vrtsvcseven t(251).vcstrap(0).normalvcsevent(2) MODULE VRTS-SNMP-MIB PARENT vcstrap PREV SIBLING errorvcsevent STATUS current OBJECTS 1. vcstag 2. vcsdate 3. vcstime 4. vcserrmsg 5. vcserrcode Appendix F, Event IDs 163
logevent logevent 6. vcshost 7. vcsres 8. vcsagent 9. vcsnode The normal VCS event with TAG_D, TAG_E. Detailed event information can be found from the objects associated with this event. TYPE NOTIFICATION-TYPE OID 1.3.6.1.4.1.1035.301.1.251.0.1 FULL PATH iso(1).org(3).dod(6).internet(1).private(4).enterprises(1).veritas(1035).commonobjects(301 ).log(1).vrtslogmonitor(251).logtrap(0).logevent(1) MODULE VRTS-SNMP-MIB PARENT logtrap STATUS current OBJECTS 1. loghost 2. logobjname 3. logfile 4. logerrstrs 5. logdesc 164 Administrators Guide
processevent processevent 6. logresolution 7. logerrcode 8. logerrmsg The sequences of specific error strings are detected in given error log file on given host. TYPE NOTIFICATION-TYPE OID 1.3.6.1.4.1.1035.301.2.251.0.1 FULL PATH iso(1).org(3).dod(6).internet(1).private(4).enterprises(1).veritas(1035).commonobjects(301 ).process(2).vrtsprocessmonitor(251).processtrap(0).processevent(1) MODULE VRTS-SNMP-MIB PARENT processtrap STATUS current OBJECTS 1. pshost 2. psid 3. psname 4. psattr 5. pserrcode 6. pserrmsg Appendix F, Event IDs 165
collectorevent collectorevent The monitored process has violated threshold condition of specified monitored attribute(existence, hanging or runaway) on given host. TYPE NOTIFICATION-TYPE OID 1.3.6.1.4.1.1035.301.4.251.0.1 FULL PATH iso(1).org(3).dod(6).internet(1).private(4).enterprises(1).veritas(1035).commonobjects(301 ).collector(4).vrtscollectormonitor(251).collectortrap(0).collectorevent(1) MODULE VRTS-SNMP-MIB PARENT collectortrap STATUS current OBJECTS 1. clname 2. clhost 3. clprdname 4. clobjname 5. clattrname 6. clerrcode 7. clerrmsg 8. clresolution 166 Administrators Guide
coldstart coldstart heartbeat The collection has detected a threshold violation condition of specified objects attribute on given host. TYPE NOTIFICATION-TYPE OID 1.3.6.1.4.1.1035.301.251.0.1 FULL PATH iso(1).org(3).dod(6).internet(1).private(4).enterprises(1).veritas(1035).commonobjects(301 ).VRTSeneMonitor(251).eneTrap(0).coldStart(1) MODULE VRTS-SNMP-MIB PARENT enetrap NEXT SIBLING heartbeat STATUS current The VERITAS SNMP Extension agent has started. TYPE NOTIFICATION-TYPE Appendix F, Event IDs 167
heartbeat OID 1.3.6.1.4.1.1035.301.251.0.3 FULL PATH iso(1).org(3).dod(6).internet(1).private(4).enterprises(1).veritas(1035).commonobjects(301 ).VRTSeneMonitor(251).eneTrap(0).heartBeat(3) MODULE VRTS-SNMP-MIB PARENT enetrap PREV SIBLING coldstart NEXT SIBLING shutdown STATUS current The VERITAS SNMP Extension agent heart beat event to signal it is alive. 168 Administrators Guide
VERITAS SNMP Event IDs in SNMP v1 VERITAS SNMP Event IDs in SNMP v1 VRTSnbuJobEvent ENTERPRISE 1.3.6.1.4.1.1035.1.1.251 SPECIFIC NUMBER abnormaljobevent (1) NormalJobEvent (2) VRTSnbuExtEvent ENTERPRISE 1.3.6.1.4.1.1035.1.101 SPECIFIC NUMBER nbuextevent(1) VRTSnbuEvents ENTERPRISE 1.3.6.1.4.1.1035.1.251 SPECIFIC NUMBER dbbackupfailed(300) dbbackupisdisabled(301) freezingmedia(302) suspendingmedia(303) mediarequiredforrestore(304) downeddrive(305) exceededcleaningfrequency(306) exceededmaxmounts(307) Appendix F, Event IDs 169
VRTSvxvmEvent mountrequest(308) nocleaningtape(309) zerocleaningsleft(310) VRTSvxvmEvent ENTERPRISE 1.3.6.1.4.1.1035.2.251 SPECIFIC NUMBER errorvxvmevent(1) normalvxvmevent(2) VRTSvxfsEvent VRTSvcsEvent ENTERPRISE 1.3.6.1.4.1.1035.3.251 SPECIFIC NUMBER errorvxfsevent(1) normalvxfsevent(2) ENTERPRISE 1.3.6.1.4.1.1035.7.251 SPECIFIC NUMBER errorvcsevent(1) normalvcsevent(2) VRTSLogMonitor ENTERPRISE 1.3.6.1.4.1.1035.301.1.251 170 Administrators Guide
VRTSProcessMonitor SPECIFIC NUMBER logevent(1) VRTSProcessMonitor ENTERPRISE 1.3.6.1.4.1.1035.301.2.251 SPECIFIC NUMBER processevent (1) VRTSCollectorMonitor ENTERPRISE 1.3.6.1.4.1.1035.301.4.251 SPECIFIC NUMBER collectorevent (1) VRTSeneMonitor ENTERPRISE 1.3.6.1.4.1.1035.301.251 SPECIFIC NUMBER coldstart(1) heartbeat(3) Appendix F, Event IDs 171
VRTSeneMonitor 172 Administrators Guide
Index Numerics 1 hanging 41 2 runaway 41 A B C D E Agent usage 46 architectural overview 4 attribute 43 BytesIO 41 CA UniCenter 23 ClassName 33 code 43 collector.conf customizing 42 Configuring frameworks 19 Configuring TEC 22 count 43 Customization collector.conf 42 freq.conf 35 messages.conf 38 notify.conf 31 process.conf 40 required 28 snmp.conf 34 snmpd.conf 28 vcs.conf 44 vxvm.conf 45 defaultport 35 defcommunity 35 defversion 35 Desc 39 Email address 9 F H I Error conditions NBU 53 VxVM 105 ErrorCode 41 Events condition based 50 critical process 50 samples 50 threshold based 50 Events, common 49 Events, types monitored 5 Features 8 filtering 36 Frameworks CA Unicenter 23 HP ITO 21 HP Openview 20 IBM 22 supported 19 TEC 22 Tivoli NetView 22 TNG Enterprise Manager 23 Frameworks, configuration 19 Frameworks, supported 7 freq 42 freq.conf customizing 35 heartbeat 36 high 43 HostName 33 HP ITO Integration 21 HP OpenView 20 IBM/Tivoli NetView 22 Installation 173
L M N O P R preliminary considerations 12 resulting files 16 support 18 system requirements 12 UNIX 14 Windows 15 lock 42 LogFileIndex 39 LogfileIndex 37 LogfilePath 37 low 43 MaxSleepTime 41 messagefreq 35 messages.conf customizing 38 mibdirs 35 MsgCode 38 names 44 NBU critical process events 55 error conditions 53 NetView 22 notify.conf customizing 31 notifyaddr 32 notifyerrcode 33 notifyobjtype 32 notifyseverity 33 notifytype 32 object 43 objectname 43 ObjType 38 PercentageCpu 41 Platforms, supported 7 Proactive event notification, diagram 5 process.conf customizing 40 processfreq 36 ProcessName 40 Resolution 39 S T U V resolution 43 result 42 samplefreq 42 script 42 SearchText 39 Severity 39 severity 43 SNMP integration, diagram 4 snmp.conf customizing 34 snmpd.conf customizing 28 snmptrap_version 36 Supported frameworks 7 Supported platforms 7 System requirements 12 tag level 45 TEC configuration 22 timeout 43 Tivoli NetView 22 TNB Enterprise Manager 23 Unicenter 23 UseDefault 39 VCS critical daemon process error conditions 90 critical events 90 error conditions Error conditions VCS 89 error conditions for Windows 102 standard error codes 90 vcs.conf customizing 44 VxFS error conditions Error conditions VxFS 61 VxVM capacity and rules related error conditions 106 error conditions 105 174 Administrators Guide
standard error codes 107 vxvm.conf customizing 45 Index 175