Active IQ Unified Manager Discussions

NetApp-Harvest 1.6.1 updater + some news about Harvest 2.0

vachagan_gratian

 

Dear Harvest users,

 

First of all, apologies for not responding to your questions lately, I was too busy but I'll try to get back to unanswered messages during the next week. Here is some good news: we released a Harvest updater to fix some issues in Harvest 1.6 and add some requested features. We don't go for an official release, since that would take a lot (more) time. The updates include:

 

  • Support for SSL authentication in Harvest Extensions
  • Fixing bug in the extension snapmirror_replications.py
  • New extension to collect capacity counters (without OCUM)
  • Caching resolved Graphite hostname (previously sending metrics to Graphite could add pressure on your DNS if caching was not configured in your server/network).

Here is how to run the updater:

  • Download the updater package here
  • Verify MD5 checksum:
$ md5sum harvest_updater_161.tar.gz 
> 1923977dee44366080ca19e724ad4650  harvest_updater_161.tar.gz
  • Unextract the package somewhere on your Harvest server, e.g.:
$ tar -xzvf harvest_updater_161.tar.gz -C /tmp/
  • Stop all harvest pollers
  • Run the updater:
$ cd harvest_updater_161/
$ ./harvest_updater
  • Restart Harvest

The updater adds three Grafana dashboards which you'll need to manually import in the Grafana webgui to use/update them:

/opt/netapp-harvest/grafana/db_netapp-detail-nfs-connections.json
/opt/netapp-harvest/grafana/db_netapp-detail-snapmirror.json
/opt/netapp-harvest/grafana/db_netapp-detail-volume-capacity.json

 

Reversing the update. Before the updater changes any files, it will create a backup in /opt/netapp-harvest/backup/harvest_updater_16100/, so if something goes wrong, you can reverse the update by:

$ ./harvest_updater --reverse

 

Second of all, many of you are asking about Harvest 2.0 and about replacing Graphite. We are well aware of the scalability issues of Graphite, and while we will continue supporting Graphite, our main backend in Harvest 2.0 will be (most likely) Prometheus. Unfortunately I can't give estimation of a release date, since at the moment we are trying to get more manpower behind this project, but I can tell you that Harvest 2.0 is our main focus at the moment.

 

Finally if by any chance you have written a Python module to send performance metrics to Prometheus and you want to contribute to an open-source project (Harvest 2.0 will be on Github!), please get in touch with me.

 

Cheers,

 

Vachagan

51 REPLIES 51

regfger

The link for harvest updater 161 is expired, can you share another valid link ?

cprivitere

@vachagan_gratian  The 1.6.1 download link is expired again. Can you please update it and the follow up patches you posted in the thread? 

unixadmins

Hi,

      can someone share the harvest 1.6.1 or it will be uploaded in the support site section https://mysupport.netapp.com/site/tools/tool-eula/netapp-harvest/download ?

 

Thanks

TorHF

Sorry, the link has expired.
The link was set to expire after a certain amount of time. Please contact the person who shared this link with you.

Any chance you can repost the files to update to 1.6.1?
Any news on release date for 1.7?

nvidia-india

Few questions here.

 

1. If backed of harvest 2.0 is moved to Prometheus, are you providing any tools to migrate existing data from whisper to Prometheus? We have metrics for last one year and don't want to miss these metrics if we plan to move to Prometheus.

 

2. Will harvest 2.0 continue to utilize/use netapp-harvest.conf file for polling clusters? Will this file continue to be source of harvest poller service? 

Hi,

 

1. we are still looking for solutions, there are some tools available for migrating from Graphite to Prometheus, but frankly, we did not had the time to investigate this thoroughly.

 

2. yes, the configuration set-up will be very similar to what we had before: one configuration file that defines your targets and exporters and template files that define what metric objects/ counters  to collect.

zsliptaii

Download expired, could you share the updater again?

prcpa8w3p

@vachagan_gratian - yep download link expired. New link please and thanks!

Hi, here is the updated link. Sorry for the delay!

@vachagan_gratian  The download link for 1.6.1 has expired again. You may want to make a full release or host the file elsewhere as this isn't working well...But regardless, can you post an updated link? Maybe also update the link in the first post so users who find the thread get a working link to start with without having to spelunk through the replies?

@cprivitere Thanks for alerting and sorry for the inconvenience. I updated the link and it should work now again.

Hey - unfortunately the link for the sm extension is expired again.

Cheers

I have "auth_type = password" in configuration, but still getting:

[WARNING] [connect_zapi] Failed to connect to NaServer: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:618)

 

For the snapmirror extension.

bkamil

@vachagan_gratian 

It seems like SVM-DR SnapMirror relationships are not supported by the SnapMirror extension and only Volume SnapMirror is supported? I'm getting a lot of this in the log file (we're purely an SVM-DR shop):

[2020-03-12 16:49:27,914] [WARNING] [poll_snapmirrors] Destination relationship with no node or volume. Skipping
[2020-03-12 16:49:27,914] [WARNING] [poll_snapmirrors] Source relationship with no volume. Skipping

 

 

vachagan_gratian

Hi bkamil,

 

Yes, that's exactly the cas.

The reason why we skip them now is that we want to summarize SnapMirrors relationships per node (to compare that against the active replications limit of each node). We can't associate an SVM(-DR)s to a Node, but we can sum up SnapMirrors under SVM as well.

 

It probably will require a 2-3 lines of additional code... So if you need that, let me know!

bkamil

@vachagan_gratian wrote:

It probably will require a 2-3 lines of additional code... So if you need that, let me know!


Yes, that would be great, thanks!

Looking forward 🙂

vachagan_gratian

I have the code, just need to test it!

@bkamil 

 

It took me a bit more than 2 lines I blame Python for all my bugs ...but here it is. Here's how to update your extension:

 

  • Download package
  • Check integrity:

 

$ md5sum snapmirror_svm_extended.tar.gz 
> c30592a26ee4b9093b29ee257729c54d  snapmirror_svm_extended.tar.gz

 

  • Extract with tar:

 

$ tar -xzvf snapmirror_svm_extended.tar.gz

 

  • Stop Harvest and copy the 2 python files to Harvest's extension directory:

 

cp python_extension_methods_v12.py snapmirror_replications.py /opt/netapp-harvest/extension/

 

  • Finally, start Harvest and manually import db_netapp-extension-snapmirror.json to your Grafana.

 

You should see new dashboards displaying SVM breakdown:

 

Screenshot from 2020-03-18 16-59-22.png

Hi there,

 

Any chance the download link could be renewed please?

 

Thanks

Dan

Sorry, updated again. Will set a reminder for myself not to forget next time!

Announcements
Register for Insight 2021 Digital

INSIGHT 2021 Digital: Meet the Specialists 2

On October 20-22, gear up for a fully digital, totally immersive virtual experience with a downright legendary lineup of world-renowned specialists. Tune in for visionary conversations, solution deep dives, technical sessions and more.

NetApp on Discord Image

We're on Discord, are you?

Live Chat, Watch Parties, and More!

Explore Banner

Meet Explore, NetApp’s digital sales platform

Engage digitally throughout the sales process, from product discovery to configuration, and handle all your post-purchase needs.

NetApp Insights to Action
I2A Banner
Public