topic Re: Update on NetApp Harvest (2.0 & 1.7) in Active IQ Unified Manager Discussions

Update on NetApp Harvest (2.0 & 1.7)

vachagan_gratian — Wed, 04 Jun 2025 11:03:51 GMT

Dear all,

It has been a while, here is finally some news about NetApp Harvest.

TL;DR:

- We are planning Harvest 2.0 alpha-test release in around a month

- Simultaneously, we will do a (probably last) legacy release, Harvest 1.7, in August.

Harvest 2.0

Things have been going a bit slower than we expected, but we made a lot of progress. As announced earlier, we've changed the whole architecture to meet two goals that are often regarded as disjoint in software engineering: high performance and high flexibility. Flexibility mainly means that it will be much easier to integrate Harvest to any system that you want to monitor (including but not limited to Ontap systems). It will be also relatively easier to feed metrics into new databases. Out of the box, we will support Prometheus. Graphite will be supported as well, but with some limitations (since Graphite only accepts numerical data).

Before we plan on the official release of Harvest 2.0, we want to do an alpha-test release for users who want to "play around" with the new software and give us feedback. Tentatively we want to do this release late July or early August (once we decide the final date, I'll post an update here).

If you are interested to get the alpha release, please send me a message with your name, email address and organization. But please note the following:

- this will be a strictly test release, not intended for production

- Harvest 2.0 requires Linux + Python3 and optionally pip to install dependencies,

- supports only CDOT systems, 7 mode will come later

- gives you approximately the same set of counters as Harvest 1.6 + extensions

- for capacity counters there is no need for OCUM/ActiveIQUM anymore

- collected counters can be sent to Prometheus or Graphite or both, but:

- there will be limited number of Grafana dashboards ready and only for Prometheus (but of course, you can create your dashboards if wished)

- Harvest 2.0 includes a self-monitoring collector, so you can see how much CPU/RAM it uses and compare to Harvest 1.6

- it will come with a developer's documentation for writing your own collectors and exporters.

If you want to test the new package, we appreciate that a lot, since feedback at this point can help us incredibly.

If you want to wait for the official (stable) release, just keep tuned. We will keep you updated as soon as we have a release date on the horizon.

Harvest 1.7

This will probably be our last legacy release which we are planning to do in August. It will mainly include bug fixes and no new major features.

Lastly, again, I need to apologize for not responding to many messages here in communities. I will try to catch up, but if you don't get a response to your issue, please DM or tag me (probably again -- sorry).

Cheers,

Vachagan

Re: Update on NetApp Harvest (2.0 & 1.7)

EWILTS_SAS — Thu, 25 Jun 2020 22:45:25 GMT

Is StorageGRID support coming later as well?

Re: Update on NetApp Harvest (2.0 & 1.7)

vachagan_gratian — Sun, 28 Jun 2020 17:12:53 GMT

yes, and I think we won't need to write a collector from scratch, we can probably just use existing tools (e.g. https://github.com/NetApp-StorageGRID/storagegrid-monitoring) as an extension or plugin.

Re: Update on NetApp Harvest (2.0 & 1.7)

zsliptaii — Mon, 06 Jul 2020 09:02:52 GMT

Will there be a builtin option to show events on graphs?

Re: Update on NetApp Harvest (2.0 & 1.7)

SimonGordon — Thu, 09 Jul 2020 15:38:13 GMT

Will there be any way to use InfluxDB as the back-end?

Re: Update on NetApp Harvest (2.0 & 1.7)

vachagan_gratian — Tue, 14 Jul 2020 09:17:01 GMT

Hi Simon,

We are working on an InfluxDB exporter as well!

Re: Update on NetApp Harvest (2.0 & 1.7)

vachagan_gratian — Tue, 14 Jul 2020 09:19:59 GMT

Hi, do you have an example of the event you would like to see? And do you just want to see the event in a graph or get alerts?

Re: Update on NetApp Harvest (2.0 & 1.7)

vachagan_gratian — Tue, 14 Jul 2020 09:22:12 GMT

Update: we have set the date for the alpha release on 27 July!

Re: Update on NetApp Harvest (2.0 & 1.7)

zsliptaii — Tue, 14 Jul 2020 09:37:41 GMT

I would like to see them on the graphs if possible, but a kind of log repository would be cool also. (Just to have everything in 1 place.)

But for example hardware fault events would be good on the performance graphs (iops drops cause the aggr is rebuilding, etc).

Re: Update on NetApp Harvest (2.0 & 1.7)

vachagan_gratian — Tue, 14 Jul 2020 10:09:33 GMT

Yes, this should be possible (I just need to find the appropriate ZAPI for disk failures/replacement). Thanks for the idea, at this point, any ideas like this are very helpful!

Re: Update on NetApp Harvest (2.0 & 1.7)

EWILTS_SAS — Tue, 14 Jul 2020 13:57:42 GMT

Death to ZAPI - I think you mean REST API 🙂

point your browser to your cluster URL and go to /docs/api/#/storage/disk_collection_get

Re: Update on NetApp Harvest (2.0 & 1.7)

vachagan_gratian — Tue, 14 Jul 2020 14:09:05 GMT

Since not all metrics are (yet) available in REST, we will continue using ZAPI for some time (which offers around 200k metrics). But yeah, I'll be writing a REST collector once I am done with ZAPI!

Re: Update on NetApp Harvest (2.0 & 1.7)

genepuppet — Wed, 15 Jul 2020 18:52:31 GMT

This is exciting! Are you planning to have the Prometheus dashboards available on https://grafana.com/grafana/dashboards ? Also curious if there is a way to disable the Grafana instance that I assume will be in the OVA as I plan to ship the metrics to Grafana Cloud.

Re: Update on NetApp Harvest (2.0 & 1.7)

vachagan_gratian — Thu, 16 Jul 2020 09:33:13 GMT

Hi,

Regarding, the OVA / NaBox, this project is maintained by @yannb, but I assume you can disable Grafana after installation (which runs as a systemd service by default).

I'll ask my PM if we can upload the dashboards to Grafana, thanks for the idea!

Re: Update on NetApp Harvest (2.0 & 1.7)

eespin — Sat, 15 Aug 2020 20:56:58 GMT

I just wanted to check in here, I'm definitely interested in Harvest 2.0 - is it available now to us eager folks 🙂

Re: Update on NetApp Harvest (2.0 & 1.7)

vachagan_gratian — Mon, 17 Aug 2020 10:07:44 GMT

Hi,

Cool, thanks for your interest, just ping me a DM if you want to be on the list.

We have a bit of delay, since we changed a few things in the implementation (e.g. we don't use the NMDSK anymore). If you are on our list, we'll keep you updated!

Re: Update on NetApp Harvest (2.0 & 1.7)

MikeSystems — Mon, 24 Aug 2020 16:56:06 GMT

Any news on harvest 2.0? We've been running 1.6 for a while now but have a substantial prometheus infrastructure so would like to move to 2.0?

Thanks,

Mike

Re: Update on NetApp Harvest (2.0 & 1.7)

vachagan_gratian — Fri, 28 Aug 2020 14:06:11 GMT

Hi Mike,

we rolled out the alpha-release internally today, next week we'll roll it out publicly (for anyone who signed up for the alpha package). I will post some more news next week here.

based on the feedback we get from the alpha-testers, we will set a date for the official beta release (which should be sometime in September).

Cheers,

Vachagan

Re: Update on NetApp Harvest (2.0 & 1.7)

JayDev — Tue, 22 Sep 2020 04:13:56 GMT

Hi Vachagan,

Is the beta version 2.0 is out? Really interested in that integration with prometheus.

Re: Update on NetApp Harvest (2.0 & 1.7)

evelez — Tue, 29 Sep 2020 13:28:32 GMT

are you still looking for Alpha testers?