Harvest support for Grafana 2.5 available!

madden · ‎2015-10-29

Hi!

Grafana released version 2.5 recently and there are some interoperability issues with dashboards provided by Harvest 1.2.2. At first glance, I discovered that some singlestat panels report a "Multiple Series Error" that looks like this:

I modifed all impacted dashboards and they are ready to go. See this blog post for how: http://blog.pkiwi.com/grafana-2-5-support-with-netapp-harvest/

Cheers,
Chris Madden

Storage Architect, NetApp EMEA (and author of Harvest)

Blog: It all begins with data

aleeyee · ‎2015-10-30

Hi Chris,

I am sure you have read it many times but thank you for the tool. I am having an issue where the Graphite/Graphana is not pulling in SVM data directly from the filer at a customer site where I installed it. It is pulling in cluster information, network etc. I suspect after reading this I now suspect it is the version of Graphite/Grafana. Where do I look to troubleshoot this at the harvest scripts or at the Graphana/Graphite.

Thanks,

Amiel

madden · ‎2015-10-30

Hi,

Thanks, and glad you like the solution!

It is very strange that cluster data can be collected and SVM data cannot. There are troubleshooting steps in Harvest Installation and Administration Guide. You can also check the logs of the poller in /opt/netapp-harvest/log, and of course verify on the Graphite server that you didn't fill up the disk!

Cheers,
Chris Madden

Storage Architect, NetApp EMEA (and author of Harvest)

Blog: It all begins with data

GMOORE · ‎2015-11-05

Chris, is there an easy way to downgrade Grafana that you are aware of? I just deployed this today and was not aware of this issue until I experienced it and seen this post.

Great session at Insight by the way!

GMOORE · ‎2015-11-05

Just in case anyone is wondering how to go from grafana 2.5 to grafana 2.1.3, I did the following.

My install is on the debian based distro Ubuntu

apt-get remove grafana

sudo apt-get install grafana=2.1.3

**If prompted with the question

"Configuration file '/etc/grafana/grafana.ini'
==> Modified (by you or by a script) since installation.
==> Package distributor has shipped an updated version.
What would you like to do about it ?

**Answer question with default to get keep the version you are installing

sudo service carbon-cache restart

sudo service grafana-server restart
sudo service apache2 restart

If following the quick start guide same command could be used to get the proper version to address the issue

sudo apt-get install grafana=2.1.3

Im no linux guru so please research before attempting any of the above. These are the steps I used so I though I would share.

James_Castro · ‎2015-12-03

Chris,

I am still getting these errors. Is there an easy fix to them? It only works if I select 1 resource. I'm on the latest versions of Harvest and Grafana.

madden · ‎2015-12-03

Hi James,

Did you update to the latest dashboards combatible with Grafana 2.5?

See this blog post for how: http://blog.pkiwi.com/grafana-2-5-support-with-netapp-harvest/

Cheers,
Chris Madden

Storage Architect, NetApp EMEA (and author of Harvest)

Blog: It all begins with data

James_Castro · ‎2015-12-03

Yes. After a quick reboot they worked. Thanks!

Hexagon · ‎2016-02-04

Thanks so much for the update, Chris. I was scratching my head trying to figure out the most sensible series aggregations when I found this thread and saw you'd already done the work. I do have one piece of feedback though; in the process of trying to figure out what aggregation I should use for the singlestat displays, I noticed this:

% for json in netapp-harvest-grafana-2.5/*.json; do echo $json; jq '.rows[].panels[]|select(.type == "singlestat")|.targets[].target|select(contains("TopResources"))' < $json; done
[…snippy snip…]
netapp-harvest-grafana-2.5/db_netapp-detail-network-lif.json
"sumSeries(highestAverage(netapp.perf.$Group.$Cluster.svm.$SVM.lif.$LIF.sent_data, $TopResources))"
"highestAverage(sumSeries(netapp.perf.$Group.$Cluster.svm.$SVM.lif.$LIF.recv_data), $TopResources)"
"sumSeries(highestAverage(netapp.perf.$Group.$Cluster.svm.$SVM.iscsi_lif.$LIF.read_data, $TopResources))"
"sumSeries(highestAverage(netapp.perf.$Group.$Cluster.svm.$SVM.iscsi_lif.$LIF.write_data, $TopResources))"
"sumSeries(highestAverage(netapp.perf.$Group.$Cluster.svm.$SVM.fcp_lif.$LIF.read_data, $TopResources))"
"sumSeries(highestAverage(netapp.perf.$Group.$Cluster.svm.$SVM.fcp_lif.$LIF.write_data, $TopResources))"
netapp-harvest-grafana-2.5/db_netapp-detail-network-port.json
"sumSeries(highestAverage(netapp.perf.$Group.$Cluster.node.$Node.eth_port.$Port.tx_bytes_per_sec, $TopResources))"
"maxSeries(highestAverage(netapp.perf.$Group.$Cluster.node.$Node.eth_port.$Port.link_pct_util, $TopResources))"
"sumSeries(highestAverage(netapp.perf.$Group.$Cluster.node.$Node.eth_port.$Port.rx_bytes_per_sec, $TopResources))"
"sumSeries(highestAverage(netapp.perf.$Group.$Cluster.node.$Node.fcp_port.$Port.write_data, $TopResources))"
"sumSeries(highestAverage(netapp.perf.$Group.$Cluster.node.$Node.fcp_port.*.read_data, $TopResources))"
[…snip some more…]

Should recv_data and read_data be different from their surrounding stats in this way? For example, it looks like the network-port.json version of read_data should use $Port instead of *, and recv_data should be the sumSeries of a highestAverage rather than the highestAverage of a sumSeries.

Thanks again for these updated dashboards; my manager is pretty pleased with all the pretty lines and green numbers!

madden · ‎2016-02-05

Hi @Hexagon

Thanks for the feedback and I confirm you did find two mistakes! I have updated them in the master dashboards and in the next Harvest release you will see them fixed there. You seem to have a good eye so if you find any other mixups please let me know!

Cheers,
Chris Madden

Storage Architect, NetApp EMEA (and author of Harvest)

Blog: It all begins with data