Active IQ Unified Manager Discussions

Removing renamed node information from harvest/grafana

LORENZO_CONTI
7,739 Views

Hi all,

 

when a cluster node is renamed, the old entry is not wiped from the node list in the "Netapp Dashboard: Cluster"
How can we fix it?

Thanks



1 ACCEPTED SOLUTION

madden
7,722 Views

Hi @LORENZO_CONTI

 

 

Each metric in Graphite correlates to a <metricname>.wsp database file on the Graphite host filesystem, and the hierarchy is built as subdirectories that ultimately contain that file.  If you want to rename any piece of the metric name a simple filesystem directory and/or database file rename will do it.  In your case you probably have metrics files with the old node name (and lots of history), and recently created metrics files with the new names (and only recent history).  If you don't care about history you can simply remove those old directories/files.  If you do care about history you can use a Graphite whisper tool to merge the datapoints: whisper-fill.py  It gets a little harder since all the metrics are in two sets of subdirectories, but a small script, or even manual creation of the SRC DST paths with cmd to cut-n-paste in Excel, shouldn't be too hard.  Also, if you want to rename something in the future you could avoid this work by stopping Harvest, doing the object rename on the cluster, rename the directories on the Graphite host, and start Harvest again.  It's not ideal but unfortunately Graphite has no API to handle renames, so filesystem renames is the only option.

 

Cheers,
Chris Madden

Storage Architect, NetApp EMEA (and author of Harvest)

Blog: It all begins with data

 

If this post resolved your issue, please help others by selecting ACCEPT AS SOLUTION or adding a KUDO or both!

View solution in original post

5 REPLIES 5

madden
7,723 Views

Hi @LORENZO_CONTI

 

 

Each metric in Graphite correlates to a <metricname>.wsp database file on the Graphite host filesystem, and the hierarchy is built as subdirectories that ultimately contain that file.  If you want to rename any piece of the metric name a simple filesystem directory and/or database file rename will do it.  In your case you probably have metrics files with the old node name (and lots of history), and recently created metrics files with the new names (and only recent history).  If you don't care about history you can simply remove those old directories/files.  If you do care about history you can use a Graphite whisper tool to merge the datapoints: whisper-fill.py  It gets a little harder since all the metrics are in two sets of subdirectories, but a small script, or even manual creation of the SRC DST paths with cmd to cut-n-paste in Excel, shouldn't be too hard.  Also, if you want to rename something in the future you could avoid this work by stopping Harvest, doing the object rename on the cluster, rename the directories on the Graphite host, and start Harvest again.  It's not ideal but unfortunately Graphite has no API to handle renames, so filesystem renames is the only option.

 

Cheers,
Chris Madden

Storage Architect, NetApp EMEA (and author of Harvest)

Blog: It all begins with data

 

If this post resolved your issue, please help others by selecting ACCEPT AS SOLUTION or adding a KUDO or both!

LORENZO_CONTI
7,684 Views

Hello Chris,

thanks for the help. 

When you say 

"Also, if you want to rename something in the future you could avoid this work by stopping Harvest, doing the object rename on the cluster, rename the directories on the Graphite host, and start Harvest again."

you mean we can simply stop the poller of the involved cluster, do the rename, and then start it again?

Cheers

Lorenzo

madden
7,683 Views

Hi @LORENZO_CONTI

 

Yes, exactly!  Harvest (by default) refreshes it's instance list every 1 hour.  So if you rename something (a lif, a vol, a node) it will be discovered within 1 hour and you'll get a new metric for the new name.  If you stop Harvest for the cluster, rename in the cluster, rename the files on the Graphite host, and start Harvest, then when it starts it discovers the new name and when it submits the metric Graphite will update the existing db files.

 

Cheers,
Chris Madden

Storage Architect, NetApp EMEA (and author of Harvest)

Blog: It all begins with data

 

If this post resolved your issue, please help others by selecting ACCEPT AS SOLUTION or adding a KUDO or both!

rcasero
4,943 Views

Chris, i renamed a SVM, i have stopped graphite, grafana and harvest (OCUM), deleted the old svm name and when I start the pollers again they reappear, how can i get rid of the old entry for good ? I see you mentioned Graphite host file, where is that exactly so I can modify it ?

 

root@nabox:/# find / -name CIFS_NFS_SVM
/opt/graphite/storage/whisper/netapp/perf/Culpeper/CULNACL01/svm/CIFS_NFS_SVM

root@nabox:/# rm -rf /opt/graphite/storage/whisper/netapp/perf/Culpeper/CULNACL01/svm/CIFS_NFS_SVM

After I start the poller again through NABOX it appears again.

root@nabox:/# find / -name CIFS_NFS_SVM
/opt/graphite/storage/whisper/netapp/perf/Culpeper/CULNACL01/svm/CIFS_NFS_SVM

 

 

Thanks,

Ralph.

vsherkar
4,053 Views

I am still having this issue and as exactly listed. I tried different ways-

 

i> Stop harvest, delete the old names of the SVMs and create new folders for the actual SVM names.

ii> Stop harvest, rename the old svm names as the new svm names...

 

However every time i tried it, within few minutes harvest with create another folder with exactly the same old name which i do not want. It even refuses to create the records for the new SVM name folder unless i tried to manually move it over to that folder. None of the records or files are created automatically under the new SVM names.

 

Clueless about whats happening, can someone help?

Public