topic netapp-harvest not respecting desired retention policies in Active IQ Unified Manager Discussions

netapp-harvest not respecting desired retention policies

Adam-Gross — Wed, 04 Jun 2025 21:55:03 GMT

My /etc/carbon/storage-schemas.conf file looks like this:

----------/

# Schema definitions for Whisper files. Entries are scanned in order,
# and first match wins. This file is scanned for changes every 60 seconds.
#
# [name]
# pattern = regex
# retentions = timePerPoint:timeToStore, timePerPoint:timeToStore, ...

# Carbon's internal metrics. This entry should match what is specified in
# CARBON_METRIC_PREFIX and CARBON_METRIC_INTERVAL settings
[netapp.capacity]
pattern = ^netapp\.capacity\.*
retentions = 15m:100d, 1d:5y

[netapp.poller.capacity]
pattern = ^netapp\.poller\.capacity\.*
retentions = 15m:100d, 1d:5y

[netapp.perf]
pattern = ^netapp\.perf\.*
retentions = 60s:35d, 5m:100d, 15m:395d, 1h:5y

[netapp.poller.perf]
pattern = ^netapp\.poller\.perf\.*
retentions = 60s:35d, 5m:100d, 15m:395d, 1h:5y

[netapp.perf7]
pattern = ^netapp\.perf7\.*
retentions = 60s:35d, 5m:100d, 15m:395d, 1h:5y

[netapp.poller.perf7]
pattern = ^netapp\.poller\.perf7\.*
retentions = 60s:35d, 5m:100d, 15m:395d, 1h:5y

# Default retention policies
#
#[carbon]
#pattern = ^carbon\.
#retentions = 60:90d
#
#[default_1min_for_1day]
#pattern = .*
#retentions = 60s:1d

/----------

Despite this, for some reason, the in force policy is [default_1min_for_1day]. I've restarted services and even gone so far as to reboot the server. Data still rolls off every 24 hours. Any help would be greatly appreciated. Thanks in advance!

Re: netapp-harvest not respecting desired retention policies

madden — Wed, 02 Mar 2016 13:33:34 GMT

Hi @Adam-Gross,

Two ideas:

1) The storage-schemas.conf file is only consulted the 1st time a metric is received and causes the creation of a .wsp file with the specified schema. Updating storage-schemas.conf will cause future created metrics to use new settings but the ones already created retain their original settings. See this post for some techniques to update to a different retention. You can also use "whisper-info.py filename.wsp", or "whisper-info filename.wsp" to see the retention archives of an existing file.

2) You have carbon installed more than once and the storage-schemas.conf file in /etc/carbon isn't the right one. If you installed from source the default location is /opt/graphite/conf/storage-schemas.conf. If you check the carbon logfile "creates.log" you can also see which storage-schemas entry is matching for a net metric create.

Cheers,
Chris Madden

Storage Architect, NetApp EMEA (and author of Harvest)

Blog: It all begins with data

If this post resolved your issue, please help others by selecting ACCEPT AS SOLUTION or adding a KUDO

Re: netapp-harvest not respecting desired retention policies

Adam-Gross — Wed, 02 Mar 2016 21:03:57 GMT

Shutting everything down, removing the whisper folders, and firing everything back up recreated files of the expected size. Thank you very much!

-Adam