Active IQ Unified Manager Discussions

NetApp-Harvest 1.6.1 updater + some news about Harvest 2.0

vachagan_gratian

 

Dear Harvest users,

 

First of all, apologies for not responding to your questions lately, I was too busy but I'll try to get back to unanswered messages during the next week. Here is some good news: we released a Harvest updater to fix some issues in Harvest 1.6 and add some requested features. We don't go for an official release, since that would take a lot (more) time. The updates include:

 

  • Support for SSL authentication in Harvest Extensions
  • Fixing bug in the extension snapmirror_replications.py
  • New extension to collect capacity counters (without OCUM)
  • Caching resolved Graphite hostname (previously sending metrics to Graphite could add pressure on your DNS if caching was not configured in your server/network).

Here is how to run the updater:

  • Download the updater package here
  • Verify MD5 checksum:
$ md5sum harvest_updater_161.tar.gz 
> 1923977dee44366080ca19e724ad4650  harvest_updater_161.tar.gz
  • Unextract the package somewhere on your Harvest server, e.g.:
$ tar -xzvf harvest_updater_161.tar.gz -C /tmp/
  • Stop all harvest pollers
  • Run the updater:
$ cd harvest_updater_161/
$ ./harvest_updater
  • Restart Harvest

The updater adds three Grafana dashboards which you'll need to manually import in the Grafana webgui to use/update them:

/opt/netapp-harvest/grafana/db_netapp-detail-nfs-connections.json
/opt/netapp-harvest/grafana/db_netapp-detail-snapmirror.json
/opt/netapp-harvest/grafana/db_netapp-detail-volume-capacity.json

 

Reversing the update. Before the updater changes any files, it will create a backup in /opt/netapp-harvest/backup/harvest_updater_16100/, so if something goes wrong, you can reverse the update by:

$ ./harvest_updater --reverse

 

Second of all, many of you are asking about Harvest 2.0 and about replacing Graphite. We are well aware of the scalability issues of Graphite, and while we will continue supporting Graphite, our main backend in Harvest 2.0 will be (most likely) Prometheus. Unfortunately I can't give estimation of a release date, since at the moment we are trying to get more manpower behind this project, but I can tell you that Harvest 2.0 is our main focus at the moment.

 

Finally if by any chance you have written a Python module to send performance metrics to Prometheus and you want to contribute to an open-source project (Harvest 2.0 will be on Github!), please get in touch with me.

 

Cheers,

 

Vachagan

50 REPLIES 50

Hi bugfinder,

 

Sorry for such a late reply. I appreciate very much your willingness to share your work (and I share your sympathy for Perl!) Is it okay if I get back to you when we do our next release? We'll be happy to include your extension in our package (just make sure to add your name as author)!

bugfinder

for the moment I have my extension scripts at https://users.suse.com/~ro/NetApp/index.html but this is just a temporary url.

vachagan_gratian

Hi, it seems like you are trying to execute the json files, which have only read permissions (and write for owner):

 

$ ls -l /opt/netapp-harvest/grafana/         
...
-rw-r--r-- 1 netapp-harvest netapp-harvest 6825 Jan 24 13:25 db_netapp-detail-nfs-connections.json
-rw-r--r-- 1 netapp-harvest netapp-harvest 7584 Jan 24 13:25 db_netapp-detail-snapmirror.json
-rw-r--r-- 1 netapp-harvest netapp-harvest 19602 Jan 24 13:25 db_netapp-detail-volume-capacity.json ...

You can either update the dashboards manually in the Grafana webgui (create -> import in main menu), or run the -import argument of netapp-manager, but you should be aware that this would overwrite anything that you might have changed in your dashboards after the install.

 

$opt/netapp-harvest.netapp-manager -import
[OK     ] Will import dashboards to [GRAFANA_SERVER]
[OK     ] Dashboard directory is [/opt/netapp-harvest/grafana]
[OK     ] Imported dashboard [db_netapp-detail-7-mode-lun.json] successfully
[OK     ] Imported dashboard [db_netapp-dashboard-cluster.json] successfully
...

If that's the case and you don't want to overwrite changes, you can temporarily remove the json's that you don't want to update from /opt/netapp-harvest/grafana/ then run the same import command.

moep

Will you release an updated RPM for 1.6.1? This hacky patching approach is really bad when it comes to version management.

vachagan_gratian

At some point we will do that, but I am not sure when exactly, because usually it takes a lot time to issue an official release.

KumarChalla

I understand Harvest 2.0 is a WIP and adopting Prometheus as a backend is a very welcome news!

 

I do have two questions about Harvest 2.0:

 

1. Will Harvest and/or the Prometheus exporter be available as a docker image? This would help in deploying it in Kubernetes

2. Will there be flexibility in having Harvest 2.0's exporter expose metrics to another Prometheus instance?

 

Appreciate your focus on getting Harvest 2.0 happen.

Hi Kumar,

 

1. Yes, that's on our schedule

2. Harvest 2.0 will architecture will allow to send each metric object to different (and multiple) DB instances. We consider also integrating with Thalos.


@vachagan_gratian wrote:

Hi Kumar,

 

1. Yes, that's on our schedule

2. Harvest 2.0 will architecture will allow to send each metric object to different (and multiple) DB instances. We consider also integrating with Thalos.


Thank you Vachagan!

 

To clarify, did you mean Thanos rather than Thalos?

yes, I meant thanos!

Announcements
NetApp on Discord Image

We're on Discord, are you?

Live Chat, Watch Parties, and More!

Explore Banner

Meet Explore, NetApp’s digital sales platform

Engage digitally throughout the sales process, from product discovery to configuration, and handle all your post-purchase needs.

NetApp Insights to Action
I2A Banner
Public