netapp-manager -export failed, can't see aything from dashboard

[EricL]

Hi all,

I installed/configured Harvest on Ubuntu14, everything works fine except I could not export the dashboads.  Here is the error:

# /opt/netapp-harvest/netapp-manager -export
[OK ] Will export dashboards from http://localhost:443
malformed JSON string, neither array, object, number, string or atom, at character offset 0 (before "\x{15}\x{3}\x{1}\x{0}...") at /opt/netapp-harvest/netapp-manager line 180.

BTW, I can inport the dashboard.

# ./netapp-manager -import
[OK ] Will import dashboards to http://localhost:443
[OK ] Imported dashboard [db_netapp-detail-lun.json] successfully
[OK ] Imported dashboard [db_netapp-detail-7-mode-disk-and-cache-layers.json] successfully
[OK ] Imported dashboard [db_netapp-detail-7-mode-network-port.json] successfully
[OK ] Imported dashboard [db_netapp-dashboard-node.json] successfully
[OK ] Imported dashboard [db_netapp-detail-network-port.json] successfully
[OK ] Imported dashboard [db_netapp-detail-harvest-poller.json] successfully
[OK ] Imported dashboard [db_netapp-detail-volume.json] successfully
[OK ] Imported dashboard [db_netapp-detail-disk-and-cache-layers.json] successfully
[OK ] Imported dashboard [db_netapp-dashboard-cluster.json] successfully
[OK ] Imported dashboard [db_netapp-dashboard-7-mode-node.json] successfully
[OK ] Imported dashboard [db_netapp-detail-7-mode-lun.json] successfully
[OK ] Imported dashboard [db_netapp.json] successfully
[OK ] Imported dashboard [db_netapp-detail-network-lif.json] successfully
[OK ] Imported dashboard [db_netapp-dashboard-cluster-group.json] successfully
[OK ] Imported dashboard [db_netapp-dashboard-svm.json] successfully
[OK ] Imported dashboard [db_netapp-dashboard-7-mode-group.json] successfully
[OK ] Imported dashboard [db_netapp-detail-7-mode-windows-file-services.json] successfully
[OK ] Imported dashboard [db_netapp-detail-7-mode-volume.json] successfully
[OK ] Imported dashboard [db_netapp-detail-graphite-server.json] successfully

Thanks in advance!

Regards,

Eric

[madden]

Hi Eric,

From the user guide about supported releases:

-------

Grafana

2.1.1
2.1.2
2.1.3
Earlier versions work but Harvest dashboard export function
is only supported from 2.1.1 onward. Import function is
supported from 2.0 onward.

------

Maybe you are running an older version of Grafana?  If yes, please update to the latest and see how that goes.  If not, please let me know versions of Grafana and Harvest and provide the export cmd output with the -v flag to get get more verbose logging.

Cheers,
Chris Madden

Storage Architect, NetApp EMEA (and author of Harvest)

Blog: It all begins with data

[EricL]

Morning Chris,

Thanks for your help. I am runing Grafana version 2.1.3 on Ubuntu 14.04

root@ubuntu14:~# dpkg -l |grep -i grafana
ii grafana 2.1.3 amd64 Grafana

Regards,

Erc

[madden]

Hi Eric,

I just tried on my Ubuntu box and it works fine so must be something about your installation.  It fetches http://SERVER_NAME:443/api/search and then parses the JSON output.  The error is actually coming from the Perl JSON module, decode_json function, so presumably the data is not valid JSON.  So that makes me curious what Grafana is returning.  

Can you visit the URL in a web browser and let me know that content?

Cheers,
Chris Madden

Storage Architect, NetApp EMEA (and author of Harvest)

Blog: It all begins with data

[EricL]

Hi Chris,

For the line https://ubuntu14.agf.ca/api/search, it just shows me "[ ]".

Actually, I don't mind repeat all installation process based on the document. it only takes lee then 2 hours. Not sure if re-install will overwrite all of old configuration.

Regards,

Eric

[madden]

Hi Eric,

Odd.  So it looks like the json is indeed empty.  Now I don't know why Grafana would return an empty response.  Uninstalling and reinstalling Grafana would be one idea (the other components like graphite and harvest don't look to blame).  You could also check the Grafana log for errors.  Also, to make sure, you do have some dashboards visible in the gui, right?  Lastly I also noticed in your original post you had http on port 443, and now in the most recent post you used https on 443.  So I'd verify that you get protocol and port correct in the netapp-harvest.conf to match your grafana config.

Cheers,
Chris Madden

Storage Architect, NetApp EMEA (and author of Harvest)

Blog: It all begins with data

[EricL]

Hi Chris,

Here is the screenshoot of Grafana.

For port:

root@ubuntu14:/opt/netapp-harvest# cat netapp-harvest.conf |grep http
grafana_url = http://localhost:443

root@ubuntu14:/opt/netapp-harvest# cat /etc/grafana/grafana.ini |grep port
# The http port to use
#http_port = 3000
http_port = 443

Regards,

Eric

[EricL]

Hi Chris,

One more thing, the log file shows me the LIsten is 0.0.0.0.443, is it correct?

2015/10/14 13:54:13 [I] Database: sqlite3, ConnectionString: file:/var/lib/grafana/grafana.db?cache=shared&mode=rwc&_loc=Local
2015/10/14 13:54:13 [I] Migrator: Starting DB migration
2015/10/14 13:54:13 [I] Listen: https://0.0.0.0:443
2015/10/14 13:56:55 [I] Starting Grafana
2015/10/14 13:56:55 [I] Version: 2.1.3, Commit: v2.1.3, Build date: 2015-08-24 04:02:44 -0400 EDT
2015/10/14 13:56:55 [I] Configuration Info
Config files:
[0]: /usr/share/grafana/conf/defaults.ini
[1]: /etc/grafana/grafana.ini
Command lines overrides:
[0]: default.paths.data=/var/lib/grafana
[1]: default.paths.logs=/var/log/grafana
Paths:
home: /usr/share/grafana
data: /var/lib/grafana
logs: /var/log/grafana

Thanks,

Eric

[madden]

Hi Eric,

You have some mixup I think.  If you want Grafana to run on HTTPS you need to also set:

[server]
# Protocol (http or https)
protocol = https

And then also configure SSL cert info:

# https certs & key file
cert_file =
cert_key =

So in the netapp-harvest.conf you have grafana on http on 443, yet in grafana he is listening on https on 443.  Also from your screenshot you have no dashboards loaded!

So you need to reconcile to use http or https consistently in grafana config and netapp-harvest.conf

I hope that makes sense!

Cheers,
Chris Madden

Storage Architect, NetApp EMEA (and author of Harvest)

Blog: It all begins with data

[EricL]

Hi Chris,

You Are the MAN! It works! You are correct. something wrong with http/https setting. I switched to HTTP and point to the port 3000. Now I can see the beautiful GUI interface. 

One more thing, for the netapp-harvest.conf file, I setup two clusters + OCUM, and I can see they are running. 

root@ubuntu14:/opt/netapp-harvest# ./netapp-manager -status
STATUS POLLER SITE
############### #################### ##################
[RUNNING] OCUM AGFOCUM
[RUNNING] netappcl01 Q9Brampton
[RUNNING] netappcl11 Q9TD

Here is the configure file regarding this section:

# [INSERT_CLUSTER_OR_CONTROLLER_NAME_HERE]
[netappcl01]
hostname = x.x.x.x
site = Q9Brampton

[netappcl11]
hostname = x.x.x.x
site = Q9TD

#====== OnCommand Unified Manager (OCUM) for cDOT capacity info ===============
#
# [INSERT_OCUM_SERVER_NAME_HERE]
[OCUM]
hostname = x.x.x.x
site = AGFOCUM
host_type = OCUM
data_update_freq = 900
normalized_xfer = gb_per_sec

But for some reason, it only showes DR site at the dashboard, there is no option to select rest of sites.

Regards,

Eric

[EricL]

Hi Chris,

Good news! I fixed the problem for production site. I can see both Pro and DR GUI from the Grafana. Do I suppose to see the OCUM menu as well?

Regards,

Eric

[madden]

Hi Eric,

Glad you were able to fix things up!

The OCUM metrics are posted by default to this hierarchy in graphite:

netapp.capacity.$Group.$Cluster.*

So $Group is your site, $Cluster is your poller name for each of your cDOT systems.  In the Grafana dashboards you will see capacity info right alongside the perf info in the various rows.  The site you put it in the OCUM poller config is only used for posting statistics on the poller itself to the netapp.poller.* graphite hierarchy.

Cheers,

Chris

Cheers,
Chris Madden

Storage Architect, NetApp EMEA (and author of Harvest)

Blog: It all begins with data

[EricL]

Hi Chris,

Really appeciate you help! I will use it as my daily monitor tool now. It has a lot of features looks like Performance Manager.

Best Regards,

Eric