How to configure Microsoft System Center Operations Manager (SCOM) 2012 R2 as SNMP trap receiver for VMware vcenter on MS Windows?

Transcription

1 How to congure Microsoft System Center Operations Manager (SCOM) 2012 R2 as SNMP trap receiver for VMware vcenter on MS Windows? One of the most interesting and very often asked things is how do I congure SCOM 2012 as a SNMP trap receiver? The solution isn't really easy and a lot of SNMP basics are needed. Again Microsoft need to do some homework to make some things much more easier and improve SCOM regarding SNMP traps. But rst I would like to thank some people for their good documentation which pushed me to the right direction: Stefan Roth Justin Cook Dave Murphy Marthijn van Rheenen rob1974 I've done some investigations of our System Center Operations Manager 2012 R2 update rollup 2 installation and here are my ndings: 1. If your SNMP trap sending device isn't able to act as an typical SNMP agent, means that there can be an SNMP daemon running, you never ever get this device discovered by SCOM! That's the fact! 2. You should force a discovery for a specic SNMP version. If the device is discovered as version 2(c) device, SCOM ignores every trap from this device if it is not a version 2 trap! To get more familiar with the SCOM SNMP network device discovery I build following scenario: Provisioning of a RedHat Enterprise Linux 5 VM and installation of the net-snmp suite. This software package delivered a full featured SNMP daemon (snmpd) and additionally a snmptrap daemon (snmptrapd). All daemons (aka services) are highly congurable and there is very good documentation. 1. Enable discovering as SNMP v1 device To act as a SNMP version 1 device is easy to congure. Just comment the v2c line. [root@<hostname> ~]# vi /etc/snmp/snmpd.conf... # groupname securitymodel securityname group notconggroup v1 notconguser #group notconggroup v2c notconguser #### # Third, create a view for us to let the group have rights to: Running the SNMP daemon We use the typical service call: [root@<hostname> ~]# service snmpd restart Stopping snmpd: [FAILED] Starting snmpd: [ OK ] [root@<hostname> ~]# If this is all done, don't forget to save and congure SCOM to detect the new network device. Don't discover it as a Linux or Unix computer! Before performing the discovery I started a port scan with nmap and found that only port 161 is opened on the SCOM 2012 installation. This is the reason why SCOM 2012 isn't able to act as a SNMP trap receiver out of the box unless the device discovery is successful done! [root@<hostname> ~]# nmap -su -p U:161 <FQDN-SCOM> Starting Nmap 4.11 ( ) at :28 CEST Interesting ports on <FQDN-SCOM> (<SCOM-IP>): PORT STATE SERVICE 161/udp open ltered snmp MAC Address: 00:50:56:A3:xx:yy (VMWare) Nmap nished: 1 IP address (1 host up) scanned in seconds [root@<hostname> ~]# [root@<hostname> ~]# nmap -su -p U:162 <FQDN-SCOM> Starting Nmap 4.11 ( ) at :28 CEST Interesting ports on <FQDN-SCOM> (<SCOM-IP>): PORT STATE SERVICE 162/udp closed snmptrap MAC Address: 00:50:56:A3:xx:yy (VMWare) Nmap nished: 1 IP address (1 host up) scanned in seconds [root@<hostname> ~]# But now let us congure the MS Windows Server 2012 and the SCOM 2012 R2 update rollup 2 in several separate steps. 1. First we have to do some prerequisites on the Windows operating system a. Click on Server Manager -> Dashboard -> Add roles and features

2 b. Read and click next c. Select role-based of feature-based installation

3 d. Select a server from the server pool e. Just click next f. Check SNMP service (should be installed)

4 g. Click on Server Manager -> Tools -> Services h. Click on SNMP Service -> Properties

5 i. Set SNMP community j. Select accept SNMP packets from any host k. SNMP Service startup type has to be automatic and Service status has to be running l. SNMP Trap startup type has to be disabled and Service status has to be stopped

6 2. Second we have to create a Management Pack rule in SCOM a. Click on Authoring -> Management Pack Objects -> Rules -> Create a new rule... b. Select SNMP Trap (Event) rule type and click on New... near Management pack c. Create the Management pack

7 Enter a useful name like "Override for SNMP Trap". Click on Next. d. Enter some knowledge (or not) and click on Create e. Back on rule type selection click on Next

8 f. Select a rule target rst g. Select Node as target

9 h. Select rule name Enter a useful rule name like "Collect all SNMP traps". Check if rule is enabled and click on Next. i. Congure the trap OIDs to collect

10 In SCOM 2012 R2 update rollup 2 you can leave the Object Identier eld empty. Click on Create. j. Enable the scope view k. Select the target "Node" Enable the target "Node". Click on OK.

11 l. Review the settings of the created rule of type node m. "Collect all SNMP traps" properties Click on Edit... near "Data sources". n. SNMP Trap Event trap provider properties The Object Identier Properties are empty. Click on Cancel. 3. Third we have to create a trap monitor in SCOM a. Click on Monitoring -> New -> Folder...

12 b. Review the settings of the created rule of type node Enter a useful folder name like "SNMP Management" and select the formerly created MP "Override for SNMP Trap". Click on OK. c. Create an Event View Click on SNMP Management -> New -> Event View d. Edit the properties of the Event View In tab "Criteria" choose "Node", select "generated by specic rules" and choose the "Collect all SNMP traps" rule. Click on OK. 4. Fourth we have to create a Run As Account a. Click on Administration -> Run As Conguration -> Create Run As Account...

13 b. Specify general account properties Choose "Community String" as account type and enter a useful display name like "SNMP Proxy". Click on Next. c. Set credentials

14 Enter the community string and click on Next. d. Select a distribution security option Choose "More secure" and click on Create. e. Prompt to create a prole 5. Fifth we have to create a Run As Prole a. Click on Administration -> Run As Conguration -> Proles -> Create Run As Prole...

15 b. Specify general prole properties Enter the same display name "SNMP Proxy" like we used for the "Run As Account". Click on Next. c. Add Run As Account

16 Select "Add", choose "SNMP Proxy" and select "All targeted objects". Click on OK. d. Create prole Click on Create. e. Warning about account distribution

17 First click on "SNMP Proxy" (green hook appeares) and then click on Close. Start of the network device discovery 1. Click on Administration -> Network Management -> Discovery Wizard... Select "Network devices" and click on Next. 2. Select a management or gateway server

18 If there is just the message that no management servers are available, please check the discovery rules! 3. Check discovery rules Go to Administration -> Network Management -> Discovery Rules. If there are all your SCOM servers listet, it is not allowed to congure any further discovery rule. You have to delete at least one rule to go further! Again I can't understand this behavior. Why isn't it possible to dene various rules and select one of them for a specic discovery? Once more a lack of functionality into SCOM. 4. Set general properties Enter the name (FQDN) of your network device, select a management server and pool. Click on Next. 5. Set general properties

19 Enter the name (FQDN) of your network device, select a management server and pool. Click on Next. 6. Select a discovery type Select "Explicit discovery" and click on Next. 7. Specify the default Run As accounts for discovery

20 Select "SNMP Proxy" as account name and click on Next. 8. Specify devices Click on "Add", enter name or IP of the device, choose "ICMP and SNMP" as access mode, select "v1 or v2" as snmp version and "SNMP Proxy" as account. Click on OK and then on Next. 9. Schedule the network discovery

21 Choose "Run the discovery rule manually". Click on Next. 10. Conrm the settings Click on Create. 11. Wait for completion

22 Select "Run the network discovery rule after the wizard is closed". Click on Close. And voila there is a new network SNMP version 1 device discovered on port 161! The information showed are mainly the content of snmpd.conf of the Linux server and not from snmptrapd.conf! Furthermore the net-snmp agent version seems to be hardcoded in SCOM, because the version displayed is older than the installed version. A nmap scan again shows that port 162 is now listening on the SCOM 2012 system. [root@<hostname> ~]# nmap -su -p U:162 <FQDN-SCOM> Starting Nmap 4.11 ( ) at :31 CEST Interesting ports on <FQDN-SCOM> (<SCOM-IP>): PORT STATE SERVICE 162/udp open ltered snmptrap MAC Address: 00:50:56:A3:xx:yy (VMWare) Nmap nished: 1 IP address (1 host up) scanned in seconds [root@<hostname> ~]#

23 Now it's time to disable (stop) the SNMP daemon on the Linux server and try to send some SNMP traps. There is a command line utility which can be used for sending SNMP traps. Don't confound things, this is not snmptrapd! [root@<hostname> ~]# service snmpd stop Stopping snmpd: [ OK ] [root@<hostname> ~]# [root@<hostname> ~]# snmptrap -v 1 -c public <FQDN-SCOM> "" 6 17 "" s "Just there" [root@<hostname> ~]# or [root@<hostname> ~]# snmptrap -v 1 -c public <FQDN-SCOM> NET-SNMP-EXAMPLES-MIB::netSnmpExampleNotication "" 6 17 "" SNMPv2-MIB::sysLocation.0 s "Just here" [root@<hostname> ~]# shows following result in SCOM. Please note that SCOM doesn't translate the OIDs to the appropriate entity names by itself! For the interested reader now a small collection of SNMP v2c and v1 traps and how SCOM detected them. Trap version 2(c) SNMP trap Use of snmptrap cli: [root@<hostname> ~]# snmptrap -v 2c -c public <FQDN-SCOM> 0 NET-SNMP-EXAMPLES-MIB::netSnmpExampleHeartbeatNotication netsnmpexampleheartbeatrate i 60 [root@<hostname> ~]# or [root@<hostname> ~]# snmptrap -v 2c -c public <FQDN-SCOM> i 60 [root@<hostname> ~]# Display into SCOM 2012: Trap version 1 SNMP trap Use of snmptrap cli: [root@<hostname> ~]# snmptrap -v 1 -c public <FQDN-SCOM> NET-SNMP-EXAMPLES-MIB::netSnmpExampleNotications "" 6 1 "" netsnmpexampleheartbeatrate i 60 [root@<hostname> ~]# or [root@<hostname> ~]# snmptrap -v 1 -c public <FQDN-SCOM> "" 6 1 "" i 60 [root@<hostname> ~]# Display into SCOM 2012: Now we see a bunch on non-speaking object identiers. There is a tool called snmptranslate into the net-snmp package which translates the OID into the entity name and vice versa: ObjectIdentier snmptranslate -m ALL Syntax Value snmptranslate -m ALL DISMAN-EVENT-MIB::sysUpTimeInstance Timeticks SNMPv2-MIB::snmpTrapOID.0 Oid NET-SNMP-EXAMPLES-MIB::netSnmpExampleHeartbeatNotication NET-SNMP-EXAMPLES-MIB::netSnmpExampleHeartbeatRate Integer 60 -

24 SNMPv2-MIB::snmpTrapEnterprise.0 Oid NET-SNMP-EXAMPLES-MIB::netSnmpExampleNotications If you use the option -m ALL snmptranslate will scan the MIB directory, RHEL default is /usr/share/snmp/mib, parse all MIBs and build the entity names out of the OIDs. I've bundled the needed MIBs into a download le [vmware-mibs.tar.gz;82 kbyte], which you can request here. More information about the latest MIBs are available from VMware: This was a short trip into the world of object identiers and entity names. Let us now switch back to real world. We are now able to discover a network device, which sends SNMP traps to SCOM But what about network devices which are unable to answer SNMP requests but able to send SNMP traps? Is it possible to monitor them with System Center Operations Manager? The answer is: yes, but not the smart way. First you have to deploy a server like the RedHat Enterprise Linux I've mentioned above and congure it identical (hostname and ip address and SNMP version) to your network device. If this Linux server was successful discovered by SCOM just remove it and substitute it with the "real" SNMP trap sending network device. This should work pretty good! But beware the circumstance that SCOM 2012 just shows the object identier and is not able to import MIBs! This is a severe shortage. I guess some guys know that the snmptrap daemon is able to forward SNMP traps. Now, maybe that it is possible to forward SNMP traps from a MS Windows server, let's say VMware vcenter Server, using this RHEL server? Well I've extend my scenario to verify this: Linux A sends a snmptrap to Linux B. Linux B is congured as a forwarder for SNMP traps and sends the trap immediatly back to Linux A which is a snmptrap receiver, too. I do this without including SCOM to verify that the setup is full functional. We can monitor the behavior of the two systems by observing the content of /var/log/messages on both servers. send trap from Linux A to Linux B: [root@<hostnamea> ~]# snmptrap -v 1 -c public <hostnameb> localhost 6 17 "" s "Just a test" [root@<hostnamea> ~]# Logle Linux B: Jun 18 14:46:04 <hostnameb> snmptrapd[4215]: :46:04 localhost.localdomain [ ] (via UDP: [<hostnamea-ip>]:38153) TRAP, SNMP v1, community public SNMPv2-SMI::internet Enterprise Specic Trap (17) Uptime: 265 days, 5:59:58.77 SNMPv2-SMI::internet = STRING: "Just a test" Logle Linux A: Jun 18 14:46:04 <hostnamea> snmptrapd[32387]: :46:04 localhost.localdomain [ ] (via UDP: [<hostnameb-ip>]:37360) TRAP, SNMP v1, community public SNMPv2-SMI::internet Enterprise Specic Trap (17) Uptime: 265 days, 5:59:58.77 SNMPv2-SMI::internet = STRING: "Just a test" Hey, that looks good. /var/log/messages from Logle Linux A: Jun 18 14:47:00 <hostnamea> snmptrapd[32387]: :47:00 <vcenter-fqdn> [<vcenter-ip>] (via UDP: [<vcenter-ip>]:56912) TRAP, SNMP v1, community public VMWARE-PRODUCTS-MIB::vmwVC Enterprise Specic Trap (VMWARE-VC-EVENT-MIB::vpxdDiagnostic) Uptime: 34 days, 21:34:31.53 Isn't it cool? It will work pretty good from the Linux point of view. Let's transfer the scenario using SCOM 2012 as SNMP trap receiver, which means that there is an additional trap receiver to congure: As you might guess, the result is: nothing! I can't see any new trap into SCOM 2012 by using the last scenario, just the old ones sending direct from Linux B. Oh, what a mess! These are the conguration changes on Linux B: 1. /etc/snmp/snmptrapd.conf authcommunity net,log public forward default <hostnamea-ip> public

25 forward default <SCOM-IP> public traphandle VMWARE-VC-EVENT-MIB::vpxdDiagnostic /root/scom/bin/traps vpxdiag traphandle VMWARE-PRODUCTS-MIB::vmwVC /root/scom/bin/traps vcenter traphandle /root/scom/bin/traps vcenteroid traphandle /root/scom/bin/traps vpxdiagoid disableauthorization yes You see the forward rules, they are self-explaining. The next lines are trap handlers, I will tell a bit later about them. The last line (disableauthorization yes) is needed if you use net-snmp v5.3 or greater. 2. Create the trap handler script /root/scom/bin/traps The trap handlers are able to execute a script if a specic trap is received. Because the forwarding doesn't work, what about triggering a new trap with the received values? Let's see what happens if we create the script. #!/bin/bash BASE='/usr/bin/snmptrap -v 1 -c public <SCOM-FQDN> "" 6' VPXDIAG='202 ""' VPXDALARMINFO='203 "" ' vars= while read oid val do OBJECT="$VPXDALARMINFO" if [ "$oid" = "<vcenter-fqdn>" ] then continue if [ "$oid" = "<UNKNOWN>" ] then continue if [ "$oid" = "UDP:" ] then continue if [ "$oid" = "SNMP-COMMUNITY-MIB::snmpTrapAddress.0" ] then continue if [ "$oid" = "SNMP-COMMUNITY-MIB::snmpTrapCommunity.0" ] then continue if [ "$oid" = "DISMAN-EVENT-MIB::sysUpTimeInstance" ] then continue if [ "$oid" = "SNMPv2-MIB::snmpTrapOID.0" ] then continue if [ "$oid" = "SNMPv2-MIB::snmpTrapEnterprise.0" ] then continue if [ "$oid" = "VMWARE-VC-EVENT-MIB::vmwVpxdTargetObjType.0" ] then if [ "$val" = "unknown" ] ; then val=1 elif [ "$val" = "host" ] ; then val=2 elif [ "$val" = "vm" ] ; then val=3 elif [ "$val" = "other" ] ; then val=4 if [ "$val" = "VMWARE-VC-EVENT-MIB::vpxdDiagnostic" ] then OBJECT="$VPXDIAG" break if [ "$vars" = "" ] then if [ "$oid" = "VMWARE-VC-EVENT-MIB::vmwVpxdTargetObjType.0" ] then vars="$oid i \"$val\"" else vars="$oid s \"$val\"" else if [ "$oid" = "VMWARE-VC-EVENT-MIB::vmwVpxdTargetObjType.0" ] then vars="$vars $oid i \"$val\"" else vars="$vars $oid s \"$val\"" done echo "trap: $1 # $BASE $OBJECT $vars" >> /root/scom/bin/snmptraphandle.txt clean_vars=$(echo $vars sed -e 's/ä/ae/g' -e 's/ö/oe/g' -e 's/ü/ue/g' -e 's/ä/ae/g' -e 's/ö/oe/g' -e 's/ü/ue/g' -e 's/ß/ss/g') eval "$BASE" "$OBJECT" "$clean_vars" 3. Changes of /etc/init.d/snmptrapd.... /etc/init.d/functions OPTIONS="-m ALL -Ls 3 -p /var/run/snmptrapd.pid"

26 #OPTIONS="-Lsd -p /var/run/snmptrapd.pid" if [ -e /etc/syscong/snmptrapd.options ]; then Restart of snmptrapd [root@<hostnamea> ~]# service snmptrapd restart Stopping snmptrapd: [ OK ] Starting snmptrapd: [ OK ] [root@<hostnamea> ~]# 5. The vpxd diagnostic trap To receive the vpxd diagnostic trap from vcenter every minute follow the kb article about how to change the default sending period of the diagnostic SNMP trap on vcenter: If you don't know how to send SNMP traps from vcenter there is a kb article, too: Yeah, we've got it! It works as expected. Now we can receive SNMP traps send by a vcenter Server running Microsoft Windows using a Linux server as a kind of trap rewriter. Now let me write some words about the conguration. You may have recognized in snmptrapd.conf the duplicate lines, e.g. traphandle VMWARE-VC-EVENT-MIB::vpxdDiagnostic /root/scom/bin/traps vpxdiag and traphandle /root/scom/bin/traps vpxdiagoid In your nal conguration you could delete one of the lines. It doesn't matter if you use OIDs or entity names. In my opinion it's better to use the entity names, because they are more "human readable" instead of the OIDs. Additionally you can delete the forward lines, they are obsolete. I was inspired from the handler script example from the net-snmp team for writing the traps shell script. The script is just a kind of lter which throws away unwanted information. It converts the ASCII string of the vmwvpxdtargetobjtype to integer again (the MIB les provide information about the needed data type) and at least it coverts the german umlauts (some guys say "funny or dotted characters") to the valid ASCII range. If you don't do that SCOM interpretes this as the binary representation of octet string as you can see in the following picture: Just by replacing the umlaut ä to "ae" the octet string get readable again: Now my dear reader is it up to you to build a free piece of software which displays the vcenter SNMP traps in a meaningful way into SCOM I've provided you with all information and tips I've currently discovered. You can download this page as pdf le [2000 kb]. Frank Ickstadt Am Königsbachtal Eppstein Germany Phone: not available frank [dot] ickstadt [at] removethis gmail [dot] com Fax: currently out of order Your browser: Netscape ; 5.0 (Windows)