The transition to NetApp MS Azure AD B2C is complete. If you missed the pre-registration, you will be invited to reigister at next log in.
Please note that access to your NetApp data may take up to 1 hour.
To learn more, read the FAQ and watch the video.
Need assistance? Complete this form and select “Registration Issue” as the Feedback Category.

Active IQ Unified Manager Discussions

Harvest not collecting FabricPool statistics for some nodes of a cluster

GregGuignard

Harvest is not collecting statistics for FabricPools from some nodes of a cluster but is collecting for other nodes. 

 

We're running Harvest 1.6 in NaBox

NAbox 2.5.1 (2019-09-08) 

Debian GNU/Linux 8 (jessie) - NAbox 2.5.1 (8.11) 

NetApp SDK v9.7P1 installed 
Harvest v1.6 installed 
Grafana v6.3.5 installed 
Graphite v1.2.0 installed 

 

The Grafana dashboards for Nodes 01-06 report object storage latency, reqs per second, throughput, success/failure count. Data for Nodes 07-10 are either blank or report 'No Data Points'. Latency, for example is from the Graphite tree: 

netapp, perf, $Group, $Cluster, node, $Node, fabricpool, average_latency, total-latency

 

No data is being collected by Harvest for any 'object_store_client_op' with an instance of '0'

 

# ./perf-counters-utility -host cluster.dns.name  -user user -pass secret -in -f object_store_client_op
## Connected successfully to 19.14.1.5 running 97P1
0: node07:kernel:0
0: node08:kernel:0
0: node09:kernel:0
0: node10:kernel:0
1: node02:kernel:1
2: node01:kernel:2
2: node04:kernel:2
2: node05:kernel:2
2: node06:kernel:2
4: node03:kernel:4

 

 

These will return the expected metrics for the expected nodes

# ./perf-counters-utility -host cluster.dns.name  -user user -pass secret -d -f object_store_client_op -n 2

# ./perf-counters-utility -host cluster.dns.name  -user user -pass secret -d -f object_store_client_op -n 4

 

This returns a syntax error

# ./perf-counters-utility -host cluster.dns.name  -user user -pass secret -d -f object_store_client_op -n 0

## Connected successfully to cluster.dns.name running 97P1
ERROR: -d requires the use of -u or -n.

Usage: perf-counters-utility -host <host> -user <user> -pass <pass> [-o|-in|-c|-d] [-f <family>] [-n <name> | -u <uuid>]

 

This same pattern repeats across multiple clusters, where any non-0 instance data is gathered and instance 0 data is not. Any assistance would be appreciated.

7 REPLIES 7

vachagan_gratian

Hi Greg,

 

Harvest does not emit counter data if the values are Nulls, most likely this is the case for your node 07-10. For the performance utility: try using uuid instead of name:

 

 

# ./perf-counters-utility -host cluster.dns.name  -user user -pass secret -d -f object_store_client_op -u node01:kernel:2

 

 

@vachagan_gratian  Thanks for the reply.

 

Harvest doesn't report any stats by UUID either ...

 

# ./perf-counters-utility -host  host -user username -pass password -d -f object_store_client_op -n node07:kernel:0
## Connected successfully to host running 97P1
API call failed with reason: For object_store_client_op object, no instances were found to match the given query

 

but ONTAP does have stats for the specific object/counter and node/instance pair.

 

dc1h201::statistics*> show -object object_store_client_op -instance 0 -counter stats -node node07 -raw true

Object: object_store_client_op
Instance: 0
Start-time: 4/28/2020 08:45:50
End-time: 4/28/2020 08:45:50
Scope: node07

    Counter                                                     Value
    -------------------------------- --------------------------------
    stats                                                           -
                           put.total                         71555288
                         put.success                         71555102
                         put.failure                              186
                           get.total                        963444227
                         get.success                        963443696
                         get.failure                              531
                        delete.total                         45992007
                      delete.success                         45992007
                          head.total                            56370
                        head.success                            56370

 

There are a total of 25 counters in ONTAP for this object and they all have data. I only selected 1 for illustration purposes.

use the flag -u instead of -n for UUIDs

Oops, the '-u node07:kernel:0' does return all the stats.

are those also non-empty as in the CLI?

Yes they are 

 

[stats]=[71555288,71555102,186,963444227,963443696,531,45997783,45997783,0,0,0,0,56370,56370,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]

 

yannb

@vachagan_gratian, I would take a look at this, but maybe you know off the top of your head ?

 

Is it possibily a bug in Harvest when -n is zero?

Announcements
NetApp on Discord Image

We're on Discord, are you?

Live Chat, Watch Parties, and More!

Explore Banner

Meet Explore, NetApp’s digital sales platform

Engage digitally throughout the sales process, from product discovery to configuration, and handle all your post-purchase needs.

NetApp Insights to Action
I2A Banner
Public