topic Re: Harvest not collecting FabricPool statistics for some nodes of a cluster in Active IQ Unified Manager Discussions

Harvest not collecting FabricPool statistics for some nodes of a cluster

GregGuignard — Wed, 04 Jun 2025 11:12:03 GMT

Harvest is not collecting statistics for FabricPools from some nodes of a cluster but is collecting for other nodes.

We're running Harvest 1.6 in NaBox

NAbox 2.5.1 (2019-09-08)

Debian GNU/Linux 8 (jessie) - NAbox 2.5.1 (8.11)

NetApp SDK v9.7P1 installed
Harvest v1.6 installed
Grafana v6.3.5 installed
Graphite v1.2.0 installed

The Grafana dashboards for Nodes 01-06 report object storage latency, reqs per second, throughput, success/failure count. Data for Nodes 07-10 are either blank or report 'No Data Points'. Latency, for example is from the Graphite tree:

netapp, perf, $Group, $Cluster, node, $Node, fabricpool, average_latency, total-latency

No data is being collected by Harvest for any 'object_store_client_op' with an instance of '0'

# ./perf-counters-utility -host cluster.dns.name -user user -pass secret -in -f object_store_client_op
## Connected successfully to 19.14.1.5 running 97P1
0: node07:kernel:0
0: node08:kernel:0
0: node09:kernel:0
0: node10:kernel:0
1: node02:kernel:1
2: node01:kernel:2
2: node04:kernel:2
2: node05:kernel:2
2: node06:kernel:2
4: node03:kernel:4

These will return the expected metrics for the expected nodes

# ./perf-counters-utility -host cluster.dns.name -user user -pass secret -d -f object_store_client_op -n 2

# ./perf-counters-utility -host cluster.dns.name -user user -pass secret -d -f object_store_client_op -n 4

This returns a syntax error

# ./perf-counters-utility -host cluster.dns.name -user user -pass secret -d -f object_store_client_op -n 0

## Connected successfully to cluster.dns.name running 97P1
ERROR: -d requires the use of -u or -n.

Usage: perf-counters-utility -host <host> -user <user> -pass <pass> [-o|-in|-c|-d] [-f <family>] [-n <name> | -u <uuid>]

This same pattern repeats across multiple clusters, where any non-0 instance data is gathered and instance 0 data is not. Any assistance would be appreciated.

Re: Harvest not collecting FabricPool statistics for some nodes of a cluster

yannb — Sat, 25 Apr 2020 18:26:18 GMT

@vachagan_gratian, I would take a look at this, but maybe you know off the top of your head ?

Is it possibily a bug in Harvest when -n is zero?

Re: Harvest not collecting FabricPool statistics for some nodes of a cluster

vachagan_gratian — Tue, 28 Apr 2020 09:58:29 GMT

Hi Greg,

Harvest does not emit counter data if the values are Nulls, most likely this is the case for your node 07-10. For the performance utility: try using uuid instead of name:

# ./perf-counters-utility -host cluster.dns.name -user user -pass secret -d -f object_store_client_op -u node01:kernel:2

Re: Harvest not collecting FabricPool statistics for some nodes of a cluster

GregGuignard — Tue, 28 Apr 2020 12:51:30 GMT

@vachagan_gratian Thanks for the reply.

Harvest doesn't report any stats by UUID either ...

# ./perf-counters-utility -host host -user username -pass password -d -f object_store_client_op -n node07:kernel:0 ## Connected successfully to host running 97P1 API call failed with reason: For object_store_client_op object, no instances were found to match the given query

but ONTAP does have stats for the specific object/counter and node/instance pair.

dc1h201::statistics*> show -object object_store_client_op -instance 0 -counter stats -node node07 -raw true Object: object_store_client_op Instance: 0 Start-time: 4/28/2020 08:45:50 End-time: 4/28/2020 08:45:50 Scope: node07 Counter Value -------------------------------- -------------------------------- stats - put.total 71555288 put.success 71555102 put.failure 186 get.total 963444227 get.success 963443696 get.failure 531 delete.total 45992007 delete.success 45992007 head.total 56370 head.success 56370

There are a total of 25 counters in ONTAP for this object and they all have data. I only selected 1 for illustration purposes.

Re: Harvest not collecting FabricPool statistics for some nodes of a cluster

vachagan_gratian — Tue, 28 Apr 2020 12:59:32 GMT

use the flag -u instead of -n for UUIDs

Re: Harvest not collecting FabricPool statistics for some nodes of a cluster

GregGuignard — Tue, 28 Apr 2020 13:02:48 GMT

Oops, the '-u node07:kernel:0' does return all the stats.

Re: Harvest not collecting FabricPool statistics for some nodes of a cluster

vachagan_gratian — Tue, 28 Apr 2020 13:15:33 GMT

are those also non-empty as in the CLI?

Re: Harvest not collecting FabricPool statistics for some nodes of a cluster

GregGuignard — Tue, 28 Apr 2020 13:38:13 GMT

Yes they are

[stats]=[71555288,71555102,186,963444227,963443696,531,45997783,45997783,0,0,0,0,56370,56370,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]