offbox_vscan not sending data to graphite.

[Deligatedgeek]

Based on the "NetApp Harvest Administration Guide" I created a new poller configuration for offbox_vscan statistics, below is the contents of the new template file.

-----------------------------------------------------Additional Template start-------------------------------------------------------------

%poller =   (

	'offbox_vscan' =>
			{ 
				counter_list     => [ qw(instance_name instance_uuid
								connections_accepted  connections_active  connections_filer_disconnected  connections_filer_disconnected_on_error  
								connections_filer_disconnected_timedout  connections_received  connections_rejected  connections_remote_disconnected  
								connections_total_disconnected  dispatch_latency  dispatch_latency_base  dispatch_latency_histogram
								process_name  scan_latency  scan_latency_base  scan_latency_histogram  scan_noti_dispatched  scan_noti_dispatched_rate  
								scan_noti_pending  scan_noti_received  scan_noti_received_rate  scan_noti_timedout  scan_on_close  
								scan_on_open  scan_on_read  scan_on_rename  scan_request_dispatched  scan_request_dispatched_rate  scan_request_pending  
								scan_request_requeued  scan_request_timedout  scan_resp_clean  scan_resp_failed  scan_resp_failed_access_denied  
								scan_resp_failed_bad_reputation  scan_resp_failed_bad_reputation_quarantined  scan_resp_failed_block  
								scan_resp_failed_block_file_access_generic  scan_resp_failed_busy  scan_resp_failed_encrypted_container  
								scan_resp_failed_expired_license  scan_resp_failed_extract_depth  scan_resp_failed_extract_size  scan_resp_failed_extract_time  
								scan_resp_failed_infected_replace_failure  scan_resp_failed_internal_error  scan_resp_failed_internal_error_block  
								scan_resp_failed_internal_timeout  scan_resp_failed_internal_timeout_block  scan_resp_failed_malformed_container  
								scan_resp_failed_no_license  scan_resp_infected_action_unknown  scan_resp_infected_deleted  scan_resp_infected_noblock  
								scan_resp_infected_quarantined  scan_resp_infected_renamed  scan_resp_infected_repair_failed  
								scan_resp_infected_repair_failed_quarantined  scan_resp_infected_repair_failed_readonly  
								scan_resp_infected_repair_quarantine_failed  scan_resp_infected_repaired  
								scan_resp_internal_error_noblock  scan_resp_internal_timeout_noblock  	
									
									) ],
				graphite_leaf    => 'svm.{instance_name}.vscan',
				enabled          => '1'
			},
};

-----------------------------------------------------Additional Template stop-------------------------------------------------------------

I added the poller as addtional template in netapp-harvest.conf

template = default,cdot-vscan.conf

When starting the worker manually with verbose it outputs the variables to the screen

./netapp-worker -poller my-nas -v
-------Lines removed-------------
[2017-02-15 11:40:08] [DEBUG  ] [offbox_vscan] Starting new poll data batch [0 to 0 of 0], batch size is [500]
[2017-02-15 11:40:08] [DEBUG  ] [offbox_vscan][6][my-svmnas-01] [connections_accepted] = [0]
[2017-02-15 11:40:08] [DEBUG  ] [offbox_vscan][6][my-svmnas-01] [connections_active] = [0]
[2017-02-15 11:40:08] [DEBUG  ] [offbox_vscan][6][my-svmnas-01] [connections_filer_disconnected] = [0]
[2017-02-15 11:40:08] [DEBUG  ] [offbox_vscan][6][my-svmnas-01] [connections_filer_disconnected_on_error] = [0]
[2017-02-15 11:40:08] [DEBUG  ] [offbox_vscan][6][my-svmnas-01] [connections_filer_disconnected_timedout] = [0]
[2017-02-15 11:40:08] [DEBUG  ] [offbox_vscan][6][my-svmnas-01] [connections_received] = [0]
[2017-02-15 11:40:08] [DEBUG  ] [offbox_vscan][6][my-svmnas-01] [connections_rejected] = [0]
-------Lines removed-------------
[2017-02-15 11:40:08] [DEBUG  ] [offbox_vscan][6][my-svmnas-01] [scan_resp_infected_repair_failed] = [0]
[2017-02-15 11:40:08] [DEBUG  ] [offbox_vscan][6][my-svmnas-01] [scan_resp_infected_repair_failed_quarantined] = [0]
[2017-02-15 11:40:08] [DEBUG  ] [offbox_vscan][6][my-svmnas-01] [scan_resp_infected_repair_failed_readonly] = [0]
[2017-02-15 11:40:08] [DEBUG  ] [offbox_vscan][6][my-svmnas-01] [scan_resp_infected_repair_quarantine_failed] = [0]
[2017-02-15 11:40:08] [DEBUG  ] [offbox_vscan][6][my-svmnas-01] [scan_resp_infected_repaired] = [0]
[2017-02-15 11:40:08] [DEBUG  ] [offbox_vscan][6][my-svmnas-01] [scan_resp_internal_error_noblock] = [0]
[2017-02-15 11:40:08] [DEBUG  ] [offbox_vscan][6][my-svmnas-01] [scan_resp_internal_timeout_noblock] = [0]
[2017-02-15 11:40:08] [DEBUG  ] [offbox_vscan] data-list poller next refresh at [2017-02-15 11:41:00]
[2017-02-15 11:40:08] [DEBUG  ] Sleeping [52] seconds

But nothing is sent to graphite, iether when running separately or when running under the manager.

I added the smb2 example to the same file and smb2 data was sent to graphite.

Why does vscan data not get sent to graphite??.

[Deligatedgeek · Accepted solution]

To answer my own question, It is working.

AV was disabled on SVM because of a performance issue, so I disabled scans on all CIFS volumes, enabled AV on the SVM and setup an OnDemand scan.  Voila Data.

I believe data is only collected if its changing.

[madden]

Hi @Deligatedgeek

Indeed, Harvest only submits non-zero data.  If it submitted zeros too there would be a lot more metrics submitted (ex: per protocol stats for every volume even though quite likely only 1 protocol is active).  Sometimes you want zeros, but usually not.

Glad you got it working and were able to add some customized counters for collection; this is exactly what I was hoping to enable with Harvest!

If you care to share your conf file and grafana dashboards I'm sure others might like to have them, or I could incorporate into a future version of Harvest.

Cheers,
Chris Madden

Solution Architect - 3rd Platform - Systems Engineering NetApp EMEA (and author of Harvest)

Blog: It all begins with data

If this post resolved your issue, please help others by selecting ACCEPT AS SOLUTION or adding a KUDO or both!

[israelmmi]

Hi,

Do you know how to get grafana to automatically refresh the views like they do in NetApp Insight?

Reuvy

[madden]

Hi @israelmmi,

To enable autorefresh just follow the 1,2,3 steps in the image below:

Cheers,
Chris Madden

Solution Architect - 3rd Platform - Systems Engineering NetApp EMEA (and author of Harvest)

Blog: It all begins with data

If this post resolved your issue, please help others by selecting ACCEPT AS SOLUTION or adding a KUDO or both!

[israelmmi]

Hi,

I think you misunderstood my question. In the Netapp insight videos (you can see them on youtube) you see them adding nodes to the workload and you see the numbers changing every second. When I configured (Last 6 hours, to now, and refresh off), that doesn't do the trick.

Reuvy

[Deligatedgeek]

Hi Reuvy,

NetApp Insight is a paid for product, this solution doesnt collect statistics that frequently.  If you need that level then you may need to get a price for Insight.

Regards,

Mark

[madden]

Hi @israelmmi

By default Harvest submits data every 60s.  You could configure refresh on the dashboard to automatically show new data points, but you would only get them every 60s.  Harvest can be configured with a more frequent polling schedule.  I use this during development (data_update_freq = 10 in the Poller config) with a smaller template file so I have quick feedback but you could also do this for demos.  Take care that you also set your Graphite carbon frequency (storage-schemas.conf) to store these granular values.  In my case I have a site called 'dev' that is configured for 10s frequency in netapp-harvest.conf, use a slimmed down collection template file for that poller, updated storage-schemas.conf for the same, and then run that poller while doing dev work. 

I hope this helps!

Cheers,
Chris Madden

Solution Architect - 3rd Platform - Systems Engineering NetApp EMEA (and author of Harvest)

Blog: It all begins with data

If this post resolved your issue, please help others by selecting ACCEPT AS SOLUTION or adding a KUDO or both!

[Deligatedgeek]

Chris is obviously correct but I would not advise doing this to your production solution unless you have a lot of disc space and compute resources.