snmp takeover snmpwalk not working

[metropolonline]

Hi there,

I am trying to trigger a snmp alarm - if a takeover has occoured.

Info:

FAS3140

Data ONTAP Release 7.2.6.

I have downloaded the MIB version for Data ONTAP 7.2.5.1 and can see this:

sysUpTime                                snmp.1.3.6.1.2.1.1.3.0
takeoverAddrs                            snmp.1.3.6.1.4.1.789.1.8.2.7.1.0
takeoverMode                             snmp.1.3.6.1.4.1.789.1.8.2.7.2.0

takeoverStatus                           snmp.1.3.6.1.4.1.789.1.8.2.7.3.0

When I try to walk takeoverStatus all I get is this:

tub01:/usr/lib/nagios/plugins# snmpwalk -v1 -c public na3c1.u.net 1.3.6.1.4.1.789.1.8.2.7.3.0
SNMPv2-SMI::enterprises.789.1.8.2.7.3.0 = INTEGER: 0
tub01:/usr/lib/nagios/plugins# snmpwalk -v1 -c public na3c1.u.net .1.3.6.1.4.1.789.1.8.2.7.3.0
SNMPv2-SMI::enterprises.789.1.8.2.7.3.0 = INTEGER: 0

Well that looks fine unless the system has actually been taken over (which is the case at this moment)

When I try to see the system uptime it all seems fine:

DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (2148005143) 248 days, 14:40:51.43

What am i doing wrong?

[msaravan · Accepted solution]

Hi,

I guess, you are talking about "cf takeover".  If that is the case, then you should look at cf table objects instead of takeoverinfo objects. The objects (takeoverAddrs, takeoverMode, takeoverStatus) you used in your SNMP query  belong to NetCache Appliance.

If you are looking at the cf settings of  your storage system, i.e equivalent to "cf status" command, then you need to query the below objects.

1: cfSettings (1.3.6.1.4.1.789.1.2.3.1)
2: cfState (1.3.6.1.4.1.789.1.2.3.2)
3: cfCannotTakeoverCause (1.3.6.1.4.1.789.1.2.3.3)
4: cfPartnerStatus (1.3.6.1.4.1.789.1.2.3.4)
5: cfPartnerLastStatusUpdate (1.3.6.1.4.1.789.1.2.3.5)
6: cfPartnerName (1.3.6.1.4.1.789.1.2.3.6)
7: cfPartnerSysid (1.3.6.1.4.1.789.1.2.3.7)
8: cfInterconnectStatus (1.3.6.1.4.1.789.1.2.3.8)

I pasted the sample walk output of one of my storage systems on cf table (1.3.6.1.4.1.789.1.2.3) below.

***** SNMP QUERY STARTED *****
1: cfSettings.0 (INTEGER) disabled(3)
2: cfState.0 (INTEGER) cannotTakeover(3)
3: cfCannotTakeoverCause.0 (INTEGER) disabledByOperator(3)
4: cfPartnerStatus.0 (INTEGER) maybeDown(1)
5: cfPartnerLastStatusUpdate.0 (TimeTicks) 146 days 07h:37m:17s.99th (1264183799)
6: cfPartnerName.0 (DisplayString) fas270-184-200
7: cfPartnerSysid.0 (INTEGER) 84189515
8: cfInterconnectStatus.0 (INTEGER) down(2)
***** SNMP QUERY FINISHED *****

Regards,

Saravanan

[metropolonline]

Hi,

Thank you for the answer. This is what I get:

na3c1

SNMPv2-SMI::enterprises.789.1.2.3.1.0 = INTEGER: 5

SNMPv2-SMI::enterprises.789.1.2.3.2.0 = INTEGER: 1

SNMPv2-SMI::enterprises.789.1.2.3.6.0 = STRING: "na3c2"

SNMPv2-SMI::enterprises.789.1.2.3.7.0 = INTEGER: 151704936

na3c2

SNMPv2-SMI::enterprises.789.1.2.3.1.0 = INTEGER: 4
SNMPv2-SMI::enterprises.789.1.2.3.2.0 = INTEGER: 4
SNMPv2-SMI::enterprises.789.1.2.3.3.0 = INTEGER: 1
SNMPv2-SMI::enterprises.789.1.2.3.4.0 = INTEGER: 3
SNMPv2-SMI::enterprises.789.1.2.3.5.0 = Timeticks: (1265719859) 146 days, 11:53:18.59
SNMPv2-SMI::enterprises.789.1.2.3.6.0 = STRING: "na3c1"
SNMPv2-SMI::enterprises.789.1.2.3.7.0 = INTEGER: 151704952
SNMPv2-SMI::enterprises.789.1.2.3.8.0 = INTEGER: 2

I know that we have a problem with one of the sides due to a takeover. But how do I catch it from the output above?

***** SNMP QUERY STARTED *****
1: cfSettings.0 (INTEGER) disabled(3)
2: cfState.0 (INTEGER) cannotTakeover(3)
3: cfCannotTakeoverCause.0 (INTEGER) disabledByOperator(3)
4: cfPartnerStatus.0 (INTEGER) maybeDown(1)
5: cfPartnerLastStatusUpdate.0 (TimeTicks) 146 days 07h:37m:17s.99th (1264183799)
6: cfPartnerName.0 (DisplayString) fas270-184-200
7: cfPartnerSysid.0 (INTEGER) 84189515
8: cfInterconnectStatus.0 (INTEGER) down(2)
***** SNMP QUERY FINISHED *****

[msaravan]

Hi ,

From your SNMP Walk output, I can understand that node na3c2 has taken over node na3c1.

The value of cfSettings.0 at node na3c1 is thisNodeDead(5). This setting indicates that this node has been taken over by its partner who is nothing but na3c2. 

The one problem I am seeing in your SNMP Walk is that, the cfInterconnectStatus.0 is down @ na3c2. I guess, the interconnect status should be up. So that, the giveback operation which will be executed in future will be succesful.

Note : I am not a CF expert. I need to explore more to confirm on this.

SNMPv2-SMI::enterprises.789.1.2.3.1.0 = INTEGER: 4
SNMPv2-SMI::enterprises.789.1.2.3.2.0 = INTEGER: 4
SNMPv2-SMI::enterprises.789.1.2.3.3.0 = INTEGER: 1
SNMPv2-SMI::enterprises.789.1.2.3.4.0 = INTEGER: 3
SNMPv2-SMI::enterprises.789.1.2.3.5.0 = Timeticks: (1265719859) 146 days, 11:53:18.59
SNMPv2-SMI::enterprises.789.1.2.3.6.0 = STRING: "na3c1"
SNMPv2-SMI::enterprises.789.1.2.3.7.0 = INTEGER: 151704952
SNMPv2-SMI::enterprises.789.1.2.3.8.0 = INTEGER: 2

The interpretation of the above output with their resolvable names is given below :

cfSettings.0 = takeoverByPartnerDisabled(4)
cfState.0  = takeover(4)
cfCannotTakeoverCause.0 = ok(1)
cfPartnerStatus.0 = dead(3)
cfPartnerLastStatusUpdate.0 = Timeticks: (1265719859) 146 days, 11:53:18.59
cfPartnerName.0 = STRING: "na3c1"
cfPartnerSysid.0 = INTEGER: 151704952
cfInterconnectStatus.0 = down(2)

Regards,

Saravanan

[metropolonline]

Hi msaravan,

Thank you so much for your help - its all working now.

Appreciate it 🙂