Suddenly failed to show iSCSI metrics in NetApp Harvest Grafana

[EdwardLeung]

Hi Support,

We are using NetApp Harvest 1.4.2 to collect ONTAP 9.5P3 storage performance metrics, it is working fine so far. The ONTAP storage was configured iSCSI, FC and NFS SVM and Grafana was able to show all such related metrics.

One day, we found that Grafana is suddenly not able to show iSCSI related metrics only. There is no error logged in netapp-harvest log file. We tried to restart both grafana server, graphite carbon-cache and netapp-harvest, it doesn't help.

Would you please advise any debug information or solution on this?

Thanks.

[serj]

Hi EdwardLeung,

Do you have one SVM with three protocols configured or separate SVM for ISCSI, NFS and so on?

Can I have a cluster name please in order to check if everything is ok from the ONTAP side?

[EdwardLeung]

Hi Serj,

I have separate SVMs for FC, NFS and iSCSI. Other SVMs don't have such problem except iSCSI SVM.

Hostname: dc7fas8281

Thanks.

[serj]

Could you please check if everything is ok with the cluster itself? The last available ASUP for node1 is from 10th of September and for node0 from 6th. Do you have approximative timestamp when you stopped to see data for ISCSI SVM?

[EdwardLeung]

Hi Serj

Thanks for your reply.

I double checked that everything is working fine in the cluster. I wonder if netapp-harvest will get null data if this iSCSI SVM is no longer in use? Because this SVM is still running and resources are still available in ONTAP cluster, but no client host is now connecting to its corresponding LIFs and no I/O.

Will it be this case if metric data is too small or no I/O and it become “null” data but not “value = 0” storing in graphite?

I stopped to see data and file update in .wsp since 17:32 18-Sep-2019 local time.

[serj]

Hi Edward,

Do you have the possibility to generate some I/O for this SVM?

[EdwardLeung]

Hi Serj,

We are not able to generate some I/O and test now, the client hosts are disconnected to iSCSI SVM. By observation, it seems netapp-harvest get null data if no I/O. May I know if anyone has the same situation? Thanks.

[vachagan_gratian]

If Harvest only get null values, it will not submit the counters (and since you see no warnings in the logs, this is likely to be the case). I would expect that you will start seeing counters again once you have some I/O on your SVM.

[EdwardLeung]

Hi vachagan,

Thanks for you reply.

Since I found there was no update to some of .swp files for metrics data in graphite since one day, I wonder if there is any problem in collecting data from ONTAP cluster. I assume all .swp files should be updated regularly if netapp-harvest is working fine even though the value if too small or no I/O.

[RajeshPanda]

@vachagan_gratian  can you help responding to the above query?