NetApp Harvest Grafana showing only 2 days of metrics

NTA1201 · ‎2019-08-14

Hello,

Grafana is showing only 2 days of metrics only, even after cofniguring the rention more than 2 days. Not sure what is the issue with the configuration. The retention doesn't update and stay to 86400 (2 days) when runing whisper-info.py command.

I followed instruction in different articles and doesn't help, can you pls help me?

https://community.netapp.com/t5/Data-Infrastructure-Management-Software-Discussions/netapp-harvest-graphs-not-showing-more-than-1-day/m-p/110826/highl...

https://community.netapp.com/t5/Data-Infrastructure-Management-Software-Discussions/NetApp-Harvest-Grafana-showing-only-7-days-of-metrics/td-p/130332

[root@]# cat /opt/graphite/conf/storage-schemas.conf
# Schema definitions for Whisper files. Entries are scanned in order,
# and first match wins. This file is scanned for changes every 60 seconds.
#
# [name]
# pattern = regex
# retentions = timePerPoint:timeToStore, timePerPoint:timeToStore, ...

# Carbon's internal metrics. This entry should match what is specified in
# CARBON_METRIC_PREFIX and CARBON_METRIC_INTERVAL settings
[carbon]
pattern = ^carbon\.
retentions = 60:90d

[netapp_perf]
pattern = ^netapp(\.poller?)?.perf7\.
retentions = 1m:35d,5m:100d,15m:395d,1h:5y

[netapp_capacity]
pattern = ^netapp(\.poller)?\.capacity\.
retentions = 15m:100d,1d:5y

[default_1min_for_1day]
pattern = .*
retentions = 60s:1d

[root@ carbon-cache-a]# /usr/bin/whisper-info.py /opt/graphite/storage/whisper/netapp/perf/SB/SBN/node/SBA1A/aggr/user_write_latency.wsp
maxRetention: 86400
xFilesFactor: 0.5
aggregationMethod: average
fileSize: 17308

Archive 0
retention: 86400
secondsPerPoint: 60
points: 1440
size: 17280
offset: 28

vachagan_gratian · ‎2019-08-19

Hi, not a Graphite expert, but I suspect that your regex patterns don't catch the counters emitted by Harvest, therefore your default retention schema is applied by Graphite (in your case the default schema is to keep counters 1 day, see the last line in your storage-schemas.conf).

So you can either change the default retention schema, or double-check the regular expressions.

vachagan_gratian · ‎2019-08-19

The regex that looks suspicous is

pattern = ^netapp(\.poller?)?.perf7\.

since this will only catch metrics collected from 7-Mode filers. I would expect it to be

pattern = ^netapp(\.poller)?\.perf7?\.

instead, since this will also catch metrics from a cDot system.

NTA1201 · ‎2019-08-22

Hello Vachagan,

Thx for you reply. I modified regex as suggested but doesn't help.

It seems that it using default value (1min 1 day) and doesn't take into account the config file storage-schemas.conf.

cat /opt/graphite/storage/log/carbon-cache/carbon-cache-a/creates.log
21/08/2019 00:20:15 :: new metric netapp.perf.MK.net.svm.net01mir.vol.m1m_netFile02_clone_20082019_140141_22.write_ops matched schema default_1min_for_1day