Active IQ Unified Manager Discussions

NetApp-Harvest 1.6 is out!

vachagan_gratian
16,916 Views

Hi Folks,

 

We have released Harvest 1.6 and it's available for download on ToolChest. There are three major changes in the new release, here's a short description of each of them:

 

Harvest Extension Manager (HEM)
We have added an entirely new component to Harvest: HEM allows you to customize and extend your poller without the need to touch Harvest source code. You can add your own scripts in any programming language and Harvest will run them during each polling session.

 

Harvest 1.6 comes with two extensions (NFS connections and SnapMirror replications collectors), but you can write your extension scripts. You might want to do that if you need additional post-processing on already collected metrics or if you want to collect metrics from some other source, or actually for any purpose. And you don't need to start from scratch: we have included in Harvest 1.6 templates for extension scripts in Python, Perl and Bash.

 

Check the NetApp Harvest Extension Manager 1.6 guideline for more info.

 

FlexGroup Capacity Metrics
Until now Harvest did not collect capacity metrics for FlexGroup volumes. This was due to ZAPI not providing sufficient information about FlexGroup volumes (e.g. the Cluster to which the volume belonged). Now we use RestAPI to get this missing data. This feature is supported for OCUM systems 9.4 or higher. By default it's disabled: see our Installation and Administration Guide to find out how to activate it.

 

Kudos to @Zoltan_Somogyvari for putting a lot of time and energy in testing this feature many many times!

 

AutoSupport Statistics
Harvest will send poller statistics to AutoSupport every month. This new feature is important to us, Harvest developers. As we move forward and want to make the future Harvest more scalable, resource-efficient and robust, and for that we want to have some statistics about the many thousand of Harvest pollers, such as average API runtimes, fails nodes monitored, etc. This feature is by default enabled, but you can disable it in Harvest's configuration file.

 

We hope you will upgrade your Harvest soon and look forward to your feedback!

18 REPLIES 18

bkamil
16,680 Views

Great news Vachagan, thanks a lot!


2 questions on SnapMirror replications collector:

- is there a corresponding Grafana Dashboard included as well?

- does the collector support SVM-DR or only volume-based SnapMirror?

 

vachagan_gratian
16,673 Views

Hi! Yes, a dashboard is included and the counters are summed up on node-level.

Chandler
16,421 Views

How can I disable the autosupports each month?  You mentioned it's enabled by default but can be disabled in the Harvest config file however I'm unable to locate that option in the config file.

 

Edit:  I believe I found the line that needs to be manually added:

send_autosupport_stats = 0

vachagan_gratian
16,321 Views

yes, that will disable it.

DPS_ICS
16,216 Views

Hi,

 

Thank you for this new version !

I'd like to use new dashboards on Grafan, but I've got an error message with HEM :

 

[root@server lib]# pwd
/opt/netapp-harvest/lib
[root@server  lib]# ls
DfmErrno.pm  NaElement.pm  NaErrno.pm  NaServer.pm  OCUMAPI.pm  OCUMClassicAPI.pm  Ontap7ModeAPI.pm  OntapClusterAPI.pm  ONTAPILogParser.pm  ONTAPITestContainer.pm  python  SdkEnv.pm  Test.pm
[root@server lib]# /opt/netapp-harvest/extension/snapmirror_replications.py -host XX.XX.XX.XX user netapp-harvest -pass password -port 443 -graphite_host localhost -graphite_port 2003 -graphite_root default -harvest_dir /opt/netapp-harvest/ -v
[2019-10-09 16:06:25,296] [ERROR] [load_prereqs] Failed to import NaServer: No module named NaServer
Traceback (most recent call last):
  File "/opt/netapp-harvest/extension/snapmirror_replications.py", line 618, in <module>
    main()
  File "/opt/netapp-harvest/extension/snapmirror_replications.py", line 89, in main
    load_prereqs()
  File "/opt/netapp-harvest/extension/snapmirror_replications.py", line 213, in load_prereqs
    'available in {}lib/. Exiting'.format(harvest_dir))
NameError: global name 'harvest_dir' is not defined

I'm with Harvest 1.6, SDK 9.5 (tested with 9.6 too), python 2.7 & perl 5. The host is a RHEL 7.7.

 

Regards,

 

Arnaud

MartINC
16,078 Views

Hi Arnaud,

 

I'm having the same issue as you.

 

NameError: global name 'harvest_dir' is not defined

vachagan_gratian
15,961 Views

Hi @DPS_ICS  and @MartINC,

 

You have a dependency issue. Be aware that if you use extensions in Harvest, they might have their own dependency requirements.

 

For this specific case, the snapmirror extension requires the Python libraries of the NetApp SDK. You already have the Perl libs from there. You need to get the latests NetApp SDK, unzip, and copy the folder python to /opt/netapp-harvest/lib/. Hope this helps!

TimBol
14,736 Views

Hi,

Any chance of adding the capability to integrate with Prometheus (https://prometheus.io). We're moving away from graphite, but we will keep Grafana, only with a Prometheus backend.

 

Cheers,

 

Guus

vachagan_gratian
14,659 Views

Hi Tim,

 

Yes, we are in the process of integrating Harvest with Prometheus. See also my post here.

israelmmi
12,910 Views

Does the 1.6 harvest support ONTAP 7.0

vachagan_gratian
12,902 Views

Hi. Do you mean 7-mode? If so, 7.3 and onwards releases are supported.

israelmmi
12,901 Views

No, I mean CDOT ONTAP 9.7. Sorry for not being clear.

vachagan_gratian
12,901 Views

yes, supported.

SChristian
12,787 Views

Hello,

I want to create a query, in Grafana, that would only display the volumes hosted by a node of the cluster, is this supported by the new Netapp Harvest 1.6 ?

jcwms1971
12,183 Views

I've been wanting to do this as well.  Have you figured out how to display volumes by node?

SChristian
6,635 Views

Unfortunately no 😞 

vachagan_gratian
6,582 Views

Sorry, I missed this question somehow earlier. Harvest creates a summary of volume metrics for each aggregate, for each node. However the actual volume metrics are only emitted under SVM's. This is only done because we don't want to store the same data twice and in the Whisper/Graphite database there is not other way around. I could modify the plugin that does this (/opt/netapp-harvest/plugin/cdot-volume) to store them under the aggregates as well if you don't mind having redundancy in your database, but I probably won't find time until end of next week.

 

This kind of stuff will be much better solved in Harvest 2.0 that will use Prometheus as its default database (Graphite/Whispler will be supported as well). In Prometheus we will not need to store the same data twice, we will just use labels to associate volumes with SVMs, nodes, aggregates, etc.

 

 

jcwms1971
6,565 Views

Thank you for the detailed response.  How long before Harvest 2.0 is released?

Having Volume stats per node or aggregate would be very helpful when trying to balance multi-node clusters.

Personally, I would not mind the duplication in the database to have this functionality now, but that may not be the case for everyone.

If you can work it in before Harvest 2.0, that would be great.  If not, I will look forward to the 2.0 release.

Thank you

Public