2017-09-14 12:46 PM
I have a Harvest 1.3 / Grafana 4.1.2 server.
I have two ONTAP 9.1P3 Clusters (FAS9000) that are giving me a 'Request Error' on the 'Cluster Dashboard'
One cluster errors on Highights / 'CPU Utlilization'
One cluster errors on Highligts / 'Disk Utilization'
This machine did run out of space at one time, I had have cleared all the zero sizes files, which resolved many other issues, but this one remains.
The response error I see on both of these is:
error: unpack requires a string argument of length 12
Research has lead me to something being currupted, but that doesn't feel right since I have other clusters that are working fine.
Has anyone run into this before?
Solved! SEE THE SOLUTION
2017-09-19 02:24 AM
This looks like a Graphite error related to corrupt db files.
Check this GitHub issue for more, including a one-liner that might identify and remove the corrupted files:
Solution Architect - 3rd Platform - Systems Engineering NetApp EMEA (and author of Harvest)
Blog: It all begins with data
If this post resolved your issue, please help others by selecting ACCEPT AS SOLUTION or adding a KUDO or both!
2017-09-19 07:27 AM
Thanks for that tip... That was one GitHub thread I did not run into in my initial google search.
I initially did not see the errors in my console.log, but after doing a restart the log started populating again. Looking at the time stamps on the affected files it appears to line up with the time we ran out of space. I cleared all those files and everything returned to normal.