Re: Update NetApp-Harvest to 1.6.1 version from 1.6

IAAS-Storage · ‎2020-04-29

Hello,

I installed the version 1.6 of NetApp Harvest but I could't make it work the new dashboards for snapmirror and nfs connections.

On the other hand I tried to update from 1.6 to 1.6.1 for getting the new capability with the added extension of capacity.

In my conf file netapp-harvest.conf I changed the next line:

template          = default

for this other:

template          = default,extension.conf

Also I create the template file extension.conf in the dir /opt/netapp-harvest/template. I have both scripts (bash and python) for gathering the information related to snapmirror and nfs connections:

/opt/netapp-harvest/extension/snapmirror_replications.py

/opt/netapp-harvest/extension/nfs-connections.sh

I've checked the logs file in both cases:

snapmirror_replications

[2020-04-29 10:59:03,225] [DEBUG] Started new session. Will poll host [10.50.0.20] for snapmirror replications                                                                                                                                                                 
[2020-04-29 10:59:03,271] [WARNING] [connect_zapi] Failed to connect to NaServer: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:618)

nfs_connections

[2020-04-29 11:00:01] [DEBUG  ] Session started                                                                                                                                                                                                                                
[2020-04-29 11:02:08] [DEBUG  ] Session ended

For add more information, I reviewed the information which harvest gather in the location /opt/graphite/storage/whisper/netapp/perf/*group*/*Cluster*/node/*nodecluster*/ and I don't have the information which grafana usess to graph the data.

total 12                                                                                                                                                                                                                                                                       
drwxr-xr-x.  3 root root   17 Apr 21 18:24 flashcache                                                                                                                                                                                                                          
drwxr-xr-x.  4 root root   40 Apr 21 18:24 ..                                                                                                                                                                                                                                  
drwxr-xr-x.  4 root root 4096 Apr 21 18:25 aggr                                                                                                                                                                                                                                
drwxr-xr-x.  8 root root   66 Apr 21 18:26 fcp_port                                                                                                                                                                                                                            
drwxr-xr-x.  2 root root  241 Apr 21 18:26 fcp                                                                                                                                                                                                                                 
drwxr-xr-x.  6 root root   46 Apr 21 18:27 hostadapter                                                                                                                                                                                                                         
drwxr-xr-x.  4 root root   35 Apr 21 18:27 headroom                                                                                                                                                                                                                            
drwxr-xr-x.  7 root root   61 Apr 21 18:29 eth_port                                                                                                                                                                                                                            
drwxr-xr-x. 14 root root  189 Apr 21 18:36 .                                                                                                                                                                                                                                   
drwxr-xr-x.  2 root root  100 Apr 21 18:42 nfsv3                                                                                                                                                                                                                               
drwxr-xr-x.  5 root root  122 Apr 21 18:44 wafl                                                                                                                                                                                                                                
drwxr-xr-x.  3 root root   81 Apr 21 18:44 processor                                                                                                                                                                                                                           
drwxr-xr-x.  2 root root 4096 Apr 21 18:44 system                                                                                                                                                                                                                              
drwxr-xr-x.  2 root root 4096 Apr 22 04:01 vol_summary

Previously of opening this issue, I have checked all topics related with this and I tried to fix it without success.

Please, could you help?

Thanks.

vachagan_gratian · ‎2020-04-29

Hi,

For the snapmirrors extension, we are aware of the issue, but haven't fixed it yet.

If you can, I would advise to follow the suggestion here and add the following lines to the snapmirror extension script:

import ssl
try:
    _create_unverified_https_context = ssl._create_unverified_context
except AttributeError:
    # Legacy Python that doesn't verify HTTPS certificates by default
    pass
else:
    # Handle target environment that doesn't support HTTPS verification
    ssl._create_default_https_context = _create_unverified_https_context

just before the main() function.

For the nfs extension, could you run it in verbose mode and see what messages you get?

IAAS-Storage · ‎2020-04-29

Hello,

Thanks for your kind and quick answer.

I did the changes, you mention in the python file: snapmirror_replications.py which is located in /opt/netapp-harvest/extension. Is it correct?

After the restart of neapp-harvest service, I got the next lines in the log file:

[2020-04-29 14:50:05,475] [DEBUG] Started new session. Will poll host [23.5.16.11] for snapmirror replications                                                                                                                               
[2020-04-29 14:50:05,636] [DEBUG] [connect_zapi] Created ZAPI with host [23.5.16.11:443], Release=NetApp Release 9.5P9: Mon Oct 28 09:18:10 UTC 2019                                                                                         
[2020-04-29 14:50:06,049] [DEBUG] [poll_snapmirrors] Collected 88 SnapMirror relationship instances                                                                                                                                          
[2020-04-29 14:50:06,085] [ERROR] [poll_nodes] ZAPI request failed: For volume object, no instances were found to match the given query.                                                                                                     
[2020-04-29 14:52:03,684] [ERROR] [poll_nodes] ZAPI request failed: For volume object, no instances were found to match the given query.

I tried to launch the bash file nfs_connections.sh before but the problem was I don´t know which information I have to add in this parameter.

      -E | --epoch            Fills environment variable _HARVEST_POLL_EPOCH

Thanks.

Florian_Mulatz · ‎2020-05-14

Hi,

similar problem on my side - NFS statistics doe not get's populated/shown in Grafana.

cheers and sorry for hijacking this post if it does not fit to the question

Update: I guess it's populated to graphite because I can see values there as seen in screenshot

Florian_Mulatz · ‎2020-05-14

Update:

It looks like that the metric which the dashboard want's to access is not populated in graphite

for example -->

aliasByNode(netapp.perf.$Group.$Cluster.node.*.nfsv3.nfs_connections, 5)

nfs_connections ... is not in graphite (screenshot attached ...) - and also the search tells me "no matching metrics"

IAAS-Storage · ‎2020-05-14

Hello Florian,

In my opinion, the problem is in the python file which doesn´t gather the nfs connections because there is some wrong in our configuration.
Maybe we did some wrong when we installed this new feature or there isn´t something well explain in the documentation file that is why I am waiting for netapp staff to provide us a respond or somebody who fixed this problem yet.

Thanks for join and give your inputs.

Florian_Mulatz · ‎2020-05-14

Hey,

could be the case yes - let's see 🙂

Hope dies in the end 🙂

Cheers