topic Re: NetApp Harvest 1.6 snapmirror and NFS-connections dashboard in Active IQ Unified Manager Discussions

NetApp Harvest 1.6 snapmirror and NFS-connections dashboard

Mytch — Fri, 08 Nov 2019 16:09:06 GMT

The NFS-connections and snapmirror dashboard is giving me this error.

Templating init failed

Datasource named ${DS_GRAPHITE} was not found

Re: NetApp Harvest 1.6 snapmirror and NFS-connections dashboard

vachagan_gratian — Fri, 08 Nov 2019 16:22:46 GMT

This might be a Grafana backward compabilitiy issue of dashboard. What version do you have? If it's older than 5, I think updating Grafana might solve this for you.

Re: NetApp Harvest 1.6 snapmirror and NFS-connections dashboard

Mytch — Fri, 08 Nov 2019 16:35:58 GMT

I will look at that. I do get these error on the extension scripts...not sure if that impacts the dashboard. I would expect not as it should show an empty one at any rate.

snapmirror

[2019-11-08 11:24:00,511] [ERROR] [poll_snapmirrors] ZAPI request failed: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:618)
[2019-11-08 11:25:01,396] [ERROR] [poll_snapmirrors] ZAPI request failed: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:618)
[2019-11-08 11:26:00,793] [ERROR] [poll_snapmirrors] ZAPI request failed: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:618)
[2019-11-08 11:27:02,743] [ERROR] [poll_snapmirrors] ZAPI request failed: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:618)

i resolved the nfs-connections one.

[2019-11-08 10:30:41] [WARNING] [sshpass] not installed, can't run package [nfs-connections.sh], exiting
[2019-11-08 10:46:01] [WARNING] [sshpass] not installed, can't run package [nfs-connections.sh], exiting
[2019-11-08 11:01:01] [WARNING] [sshpass] not installed, can't run package [nfs-connections.sh], exiting
[2019-11-08 11:15:59] [WARNING] [sshpass] not installed, can't run package [nfs-connections.sh], exiting
[2019-11-08 11:31:03] [DEBUG ] Session started
[2019-11-08 11:31:03] [DEBUG ] Session ended

Re: NetApp Harvest 1.6 snapmirror and NFS-connections dashboard

vachagan_gratian — Fri, 08 Nov 2019 16:48:15 GMT

The Dashboard import error is unrelated to those ones.

Re: snapmirror

Are you using SSL authentication? Unfortunately that's not supported in the Extention Manager (although we are planning to support that sometime soon). For now you can use the extension packages user/password authentication.

Re: nfs-connections

Seems like the dependency sshpass is missing. (`apt install sshpass` will install it on a Debian-like system). If you already installed it, it might be the same authentication issue as above.

Re: NetApp Harvest 1.6 snapmirror and NFS-connections dashboard

Mytch — Fri, 08 Nov 2019 16:59:42 GMT

As we jump topics......

I ugraded to Grafana 6.4.4 and I still get the same error. Like the variable DS_GRAPHITE is not confgiured somewhere?

I am not using SSL authentication for any part of the Harvest environment. Seems like a python thing

#====== Polled host setup defaults ============================================
host_type = FILER
host_port = 443
host_enabled = 1
template = default
data_update_freq = 60
ntap_autosupport = 0
latency_io_reqd = 10
auth_type = password

Re: NetApp Harvest 1.6 snapmirror and NFS-connections dashboard

vachagan_gratian — Wed, 13 Nov 2019 12:25:11 GMT

Seems like I did something wrong when exporting the SnapMirror dashboard. The ${DS_GRAPHITE} is a variable name from my Grafana configuration and is not recognized by your Grafana server.

Here is a quick fix you can use:

- Navigate to the SnapMirror Replications dashboard and enter Dashboard settings (click on the gear icon on right top),

- Go to JSON Model and copy the code to a text editor

- Replace all "datasource": "${DS_GRAPHITE}" with "datasource": null

- Copy the code back, save and reload the dashboard.

cc @yannb

Re: NetApp Harvest 1.6 snapmirror and NFS-connections dashboard

Mohanz — Thu, 14 Nov 2019 16:49:13 GMT

I upgraded grafana to 6.4 and harvest to 1.6, but not able to see any data for nfs connections .

I followed your post and changed datasource to null for both snapmirror relationships and nfs3 connections

nfssnapmirror

Re: NetApp Harvest 1.6 snapmirror and NFS-connections dashboard

vachagan_gratian — Fri, 15 Nov 2019 14:34:52 GMT

Can you check the log files of these two extensions? If there isn't much there, can you run them in verbose mode and check the logs again.

To run in verbose mode, just restart the poller for which you have activated in these extensions with -v, e.g.:

./netapp-manager -restart -poller POLLER -v

Btw, I found a bug in the snapmirrors extensions (making it collect only destination metrics), I'll fix it and post an updated version here soon.

Re: NetApp Harvest 1.6 snapmirror and NFS-connections dashboard

ClaudiuRasputneac — Fri, 29 Nov 2019 12:53:00 GMT

Hello,

Have you managed to use the Snapmirror and NFS Connection dashboards ?

Thanks !

Re: NetApp Harvest 1.6 snapmirror and NFS-connections dashboard

Cbrown — Fri, 13 Dec 2019 21:54:15 GMT

I'm also hitting this issue with NFS Connections and Snapmirror Replications.

Any updates?

Chris

Re: NetApp Harvest 1.6 snapmirror and NFS-connections dashboard

DANTAYLOR80 — Thu, 02 Jan 2020 14:23:07 GMT

Same issue.

Re: NetApp Harvest 1.6 snapmirror and NFS-connections dashboard

mmorshed11 — Tue, 28 Jan 2020 04:41:08 GMT

Datasource named ${DS_GRAPHITE} was not found

same issue any suggestion ?

Re: NetApp Harvest 1.6 snapmirror and NFS-connections dashboard

vachagan_gratian — Tue, 28 Jan 2020 14:27:26 GMT

Hi,

You can either run the updater here and use the new Grafana dashboards or follow my instructions here to fix the dashboard manually. Hope that helps!

Re: NetApp Harvest 1.6 snapmirror and NFS-connections dashboard

ClaudiuRasputneac — Tue, 28 Jan 2020 15:00:31 GMT

Hi,

I believe that the link on how to manually fix the dashboard is wrong. Can you please advise if this is the case and update it with the correct one.

Thanks.

Re: NetApp Harvest 1.6 snapmirror and NFS-connections dashboard

mmorshed11 — Wed, 29 Jan 2020 04:01:09 GMT

Grafana v6.5.3 Harvest 1.6

Thanks, @vachagan_gratian I was able to sort out the issue by following steps

1. Copy python module

Unzip netapp-manageability-sdk-9.7.zip (/tmp/netapp-manageability-sdk-9.7/lib/python/NetApp)
Copy these files NaElement.py NaErrno.py NaServer.py to /opt/netapp-harvest/lib

2. Update poller

# csgxxxxxxxx
[csgxxxxxx]
hostname = XXX.XX.XX.XX
group = XXX_cDot
template = default,extension.conf

3. Ignore SSL validation # due to error ZAPI request failed: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:618)

vi /opt/netapp-harvest/extension/snapmirror_replications.py

Add following red line after import logging & before def main();

import logging

import ssl

try:
_create_unverified_https_context = ssl._create_unverified_context
except AttributeError:
# Legacy Python that doesn't verify HTTPS certificates by default
pass
else:
# Handle target environment that doesn't support HTTPS verification
ssl._create_default_https_context = _create_unverified_https_context

def main():

4. Add privileges # ZAPI request failed: Insufficient privileges: user 'netapp-harvest' does not have read access to this resource

Add snapmirror show command to role

security login role create -role netapp-harvest-role -access readonly -cmddirname "snapmirror show"

5. Change all "datasource": null, from Snapmirror Replication Settings - JSON Model

${DS_GRAPHITE} to "datasource": null

Then Refresh Browser

Re: NetApp Harvest 1.6 snapmirror and NFS-connections dashboard

FRAZERBRADBURN — Wed, 29 Jan 2020 08:11:22 GMT

Going to try the above fix but the initial fix provided certainly doesn't work.

On saving the dashboard you get Datasource named null was not found when you make the initial fix.

Re: NetApp Harvest 1.6 snapmirror and NFS-connections dashboard

FRAZERBRADBURN — Wed, 29 Jan 2020 08:26:05 GMT

hey - just to add to my comment below the manual updated also didn't fix this issue.

Re: NetApp Harvest 1.6 snapmirror and NFS-connections dashboard

mmorshed11 — Wed, 29 Jan 2020 10:54:02 GMT

Make sure in netapp-harvest.conf template = default,extension.conf (not extensions.conf)

and in JSON file

all 5 entries

"datasource": null,

{
"aliasColors": {},
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": null,
"fill": 1,
"fillGradient": 0,
"gridPos": {
"h": 9,
"w": 23,
"x": 0,
"y": 0
},

Restart poller and checked the log

[root@XXXXXX log]# /opt/netapp-harvest/netapp-manager -restart -v
[OK ] Line [29] is Section [global]
[OK ] Line [30] in Section [global] has Key/Value pair [grafana_api_key]=[**********]
[OK ] Line [31] in Section [global] has Key/Value pair [grafana_url]=[https://X.X.X.X]
[OK ] Line [36] is Section [default]
[OK ] Line [37] in Section [default] has Key/Value pair [graphite_server]=[X.X.X.X]
[OK ] Line [38] in Section [default] has Key/Value pair [username]=[netapp-harvest]
[OK ] Line [39] in Section [default] has Key/Value pair [password]=[**********]
[OK ] Line [42] in Section [default] has Key/Value pair [send_autosupport_stats]=[1]
[OK ] Line [68] is Section [CLUSTERNAME]
[OK ] Line [69] in Section [CLUSTERNAME]] has Key/Value pair [hostname]=[X.X.X.X]
[OK ] Line [70] in Section [CLUSTERNAME]] has Key/Value pair [group]=[XXX_cDot]
[OK ] Line [71] in Section [CLUSTERNAME]] has Key/Value pair [username]=[netapp-harvest]
[OK ] Line [72] in Section [CLUSTERNAME]] has Key/Value pair [password]=[**********]
[OK ] Line [73] in Section [CLUSTERNAME] has Key/Value pair [template]=[default,extension.conf]

cat /opt/netapp-harvest/log/FILERNAME_netapp-harvest_netapp-harvest_snapmirror_replications.log

Re: NetApp Harvest 1.6 snapmirror and NFS-connections dashboard

vachagan_gratian — Thu, 30 Jan 2020 10:27:12 GMT

@mmorshed11 thanks a lot for this info! I forgot to mention that you have to extend the ONTAP user privileges (the bad habit of always testing with an admin user...).

The reason you only see destination snapmirrors in your Grafana is a bug in the extension. I recommend getting the recent updater to fix this (this will create a backup and overwrite your modified extension).

As for 3, I am surprised you go this error, what Python version was running the process?

Re: NetApp Harvest 1.6 snapmirror and NFS-connections dashboard

vachagan_gratian — Thu, 30 Jan 2020 10:30:53 GMT

Hi, I would recommend to use the dashboards from the updater package. Just first run the updater, then import the db_netapp-detail-snapmirror.json dashboard from the folder /opt/netapp-harvest/grafana/ to your Grafana webgui.