The transition to NetApp MS Azure AD B2C is complete. If you missed the pre-registration, you will be invited to reigister at next log in.
Please note that access to your NetApp data may take up to 1 hour.
To learn more, read the FAQ and watch the video.
Need assistance? Complete this form and select “Registration Issue” as the Feedback Category.

ONTAP Discussions

Tracking Snapmirror Transfer Times - Forwarding Snapmirror Logs to a Central Log Correlation System (syslog Splunk)

jurphville

We utilize Splunk to aggregate logs accross our enterprise.  Currently my Netapps are using syslog to forward messages/console logs, but I would also like to get snapmirror logs.  Is this something that can be done via syslog?  Is there another way that someone is accomplishing this?

My current syslog.conf file is...

*.info        /dev/console

*.info        /etc/messages

*.info        @mySplunkServer

I'm most concerned w/ being able to track snapmirror transfer times to be sure we are within our data protection policies.  What other methods are people using to accomplish this?

4 REPLIES 4

pascalduk

I do not have actual hands on experience with splunk myself, but I am told that splunk can import (log) files. With this method you could import the /etc/log/snapmirror log file from the root volume. Problem will be the format of the snapmirror log file as each individual transfer has for the request, actual start and end a seperate entry. I doubt splunk can be used to analyze this.

An option would be to use the snapmirror log parser from the netapp utility toolchest (https://now.netapp.com/eservice/toolchest?toolid=422)  to summarize the data for you first and import the results in splunk.

jurphville

You are correct, Splunk can ingest log files, and that is an option, but I have been hesitant to do that because I don't want to introduce a process that could "mess" w/ the Netapps/Snapmirror.  Splunk *should* be able to read the log files w/o disruption, but I'd love to hear from anyone who is already doing that w/ success before implementing.  For me, the ideal scenario would be where the Netapp handles to the log forwarding through a built in mechanism, rather than a third party action.

To your point about multiple logs per transfer, Splunk actually has a built in mechanism for handling that.  I can give Splunk the parameters that define a transaction (Start="Request" End="End" by Filer:Vol) and Splunk will dynamically combine the multiple events into a single transaction and automatically calculate the duration of the transfer.  I can then pull out statistics, create graphs, and combine that into a dashboard.

I have seen the Snapmirror Log Parser, but that script requires that the logs are already local to the script, and doesn't address how the logs are pulled out of the Netapp.  In that case, I would assume a basic copy/paste.

pascalduk

There is unfortunaly no other option that read the snapmirror log file, but I am pretty sure that it will not be a problem. I have my own scripts processing netapp log files for years now without any problem.

jurphville wrote:

To your point about multiple logs per transfer, Splunk actually has a built in mechanism for handling that.  I can give Splunk the parameters that define a transaction (Start="Request" End="End" by Filer:Vol) and Splunk will dynamically combine the multiple events into a single transaction and automatically calculate the duration of the transfer.  I can then pull out statistics, create graphs, and combine that into a dashboard.

That is very interesting as we probably implement splunk at my company next year and this will help me get rid off custom scripts to create the graphs of transfer times and amount transferred.

aborzenkov
There is unfortunaly no other option that read the snapmirror log file

Well ... there still exists SNMP and API to pull data from filer. Actually, API is pobably the only way to monitor current transfer progress, because log is written only after transfer is completed (basically, it corresponds to "last transfer" in snapmirror status) and SNMP does not seem to provide "progress" value.

Announcements
NetApp on Discord Image

We're on Discord, are you?

Live Chat, Watch Parties, and More!

Explore Banner

Meet Explore, NetApp’s digital sales platform

Engage digitally throughout the sales process, from product discovery to configuration, and handle all your post-purchase needs.

NetApp Insights to Action
I2A Banner
Public