ONTAP Discussions

Looking for some help on syslog configuration

ssbn743
486 Views

We have 4 NetApp's, all version 9.16.  These are government systems, so we have no way to upload logs or configs, we'll have to do this the hard way.
3 of the 4 simply will not send syslogs to the internal syslog server.  As far as we can tell, the 1 NetApp that is working, is configured exactly the same way as the 3 that are not working.

We have been through several guides and posts on the Google machine and have come up empty on everything tried.

 - We have a working filter, tested with the 'event filter test' command
- We have valid syslog destinations in IP address format (though, just to be 100% sure, we also have DNS configured and working)
- The syslog destinations have the correct filters applied
- We can generate a test event and see it in the ONTAP event log (we've been using monitor.volume.nearfull)
- Using the event history show -destination syslog_1 (syslog_1 being our defined dest) we see absolutely nothing
- This is confirmed with a tcpdump command on the syslog server itself seeing no packets
- It's as if the syslog service never gets notified that it needs to send a syslog

- We can ping and traceroute the syslog IP address (and even the DNS name) from the ONTAP CLI

At this point, we're down to a suggestion to login to the systemshell and reset notifiyd.  We are, however, pretty nervous about doing so, and since 3 of our 4 devices don't work, it seems like this is not the right thing to be mucking with.  

Does anyone have anything on this topic.  syslog configurations are pretty darn simple, usually, and ONTAP9 doesn't really seem to be any different.  Is there some obscure option, somewhere, that needs to be enabled or something?

1 ACCEPTED SOLUTION

Sanaman
345 Views

Have you checked the network interface service policies on 3 clusters don't send to syslog server. Please with the working one.  If syslog traffic is sent via management interface, then management interface must have right service policy applied and rules (FW) applied.

 

network interface show -fields service-policy -vserver <your_cluster>

Then check the returned sevice-policy has syslog-forwarding policy applied

network interface service-policy show -vserver <your_cluster>

 

Sample out put will be like this (includes syslog forwarding)

 

<your_cluster>

                             default-management management-core: 0.0.0.0/0
management-autosupport: 0.0.0.0/0
management-ssh: 0.0.0.0/0
management-https: 0.0.0.0/0
management-ems: 0.0.0.0/0
management-ntp-client: 0.0.0.0/0
management-dns-client: 0.0.0.0/0
management-ad-client: 0.0.0.0/0
management-ldap-client: 0.0.0.0/0
management-nis-client: 0.0.0.0/0
management-http: 0.0.0.0/0
backup-ndmp-control: 0.0.0.0/0
management-snmp-server: 0.0.0.0/0
management-ntp-server: 0.0.0.0/0
management-log-forwarding: 0.0.0.0/0

 

View solution in original post

4 REPLIES 4

cruxrealm
392 Views

You can also test the notification destination using the command:
event notification destination check -destination-name syslog_1

You can also try to create a different destination (i.e email) and use it to test.

If using different destination works,  then it will be connectivity issue between the storage and syslog server where you can use netapp tcpdump to debug.

ssbn743
315 Views

The check -destination only works for tcp connections.  I didn't specify in my post, but we are trying to use udp, though, I think we could do tcp if it comes to it.

Sanaman
346 Views

Have you checked the network interface service policies on 3 clusters don't send to syslog server. Please with the working one.  If syslog traffic is sent via management interface, then management interface must have right service policy applied and rules (FW) applied.

 

network interface show -fields service-policy -vserver <your_cluster>

Then check the returned sevice-policy has syslog-forwarding policy applied

network interface service-policy show -vserver <your_cluster>

 

Sample out put will be like this (includes syslog forwarding)

 

<your_cluster>

                             default-management management-core: 0.0.0.0/0
management-autosupport: 0.0.0.0/0
management-ssh: 0.0.0.0/0
management-https: 0.0.0.0/0
management-ems: 0.0.0.0/0
management-ntp-client: 0.0.0.0/0
management-dns-client: 0.0.0.0/0
management-ad-client: 0.0.0.0/0
management-ldap-client: 0.0.0.0/0
management-nis-client: 0.0.0.0/0
management-http: 0.0.0.0/0
backup-ndmp-control: 0.0.0.0/0
management-snmp-server: 0.0.0.0/0
management-ntp-server: 0.0.0.0/0
management-log-forwarding: 0.0.0.0/0

 

ssbn743
314 Views

Yes, thank you - this is definitely the problem.  We had a rouge service-policy in there (for reasons I wish I could tell you) and changing that to the default-management, which is 0.0.0.0/0 on everything, and I immediately start seeing traffic on the tcpdump on the syslog server.

Now, I can't get syslog to actually log the traffic, but that's another problem - packets are now getting there so thank you very much.

Public