netapp-harvest ocum capacity metrics

[SrikanthReddy]

I am not seeing capacity metrics from ocum into netapp-harvest db 

if any other config needs to be added,checked  for ocum other than below as in netapp-harvest document 

updated our environment info for below in our netapp-harvest.conf
[INSERT_OCUM_HOSTNAME_HERE]
hostname = INSERT_IP_ADDRESS_OR_HOSTNAME_OF_OCUM_HOSTNAME
site = INSERT_SITE_IDENTIFIER_HERE
host_type = OCUM
data_update_freq = 900
normalized_xfer = gb_per_sec
template = ocum-opm-hierarchy.conf
graphite_root = netapp-capacity.Clusters.{display_name}
graphite_meta_metrics_root = netapp-capacity-poller.INSERT_OCUM_HOSTNAME_HERE

from logs see this for OCUM
[2015-10-02 10:59:08] [WARNING] [sysinfo] Discovered [CDOT-1] on OCUM server with no matching conf section; to collect this cluster please add a section
[2015-10-02 10:59:08] [WARNING] [sysinfo] Discovered [CDOT-2] on OCUM server with no matching conf section; to collect this cluster please add a section

[J_curl]

you also need to add a section in the .conf for the cluster you are monitoring.  The "site" value ties OCUM to the cluster.  Example:

#====== 7DOT (node) or cDOT (cluster LIF) for performance info ================
 [my-clst-01]
 hostname       = 10.0.0.0
 site           = GCC

#====== OnCommand Unified Manager (OCUM) for cDOT capacity info ===============
 [ocum-gcc]
 hostname          = 10.0.0.1
 site              = GCC
 host_type         = OCUM
 data_update_freq  = 900
 normalized_xfer   = gb_per_sec

[...]

[SrikanthReddy]

i had different names for sites as clusters were in different locations  updated site to same name for al 

after restart I am still getting no matching conf section in the logs 

if you added one cluster for perf  or twice once for ocum capacity and other time for perf ? 

 [my-clst-01]
 hostname       = 10.0.

[SrikanthReddy]

one of the reason for my issue was I used hostname.subdomain.com  dns name instead of just hostname in graphite config [hostname]

fixed this and then it worked 

Thanks for help 

[madden]

Hi,

If using Harvest for Capacity and Performance the metrics hierarchies are adjacent to each other, on my system from a linux perspective:

/opt/graphite/storage/whisper/netapp/capacity/nl/sdt-cdot1/node/sdt-cdot1-01/aggr/n01fc01/

/opt/graphite/storage/whisper/netapp/perf/nl/sdt-cdot1/node/sdt-cdot1-01/aggr/n01fc01/

To build that hierarchy Harvest needs to know the clustername and the site for all monitored systems.

For Perf it uses the poller name (the one in the section header []) and the site name.

With the OCUM integration you add the OCUM server itself and Harvest then learns all clusternames monitored by OCUM.  Harvest then looks for a poller section that matches the clustername to find the site to post the capacity metrics.

So the recommended way to do it is to have the cDOT poller section name match the clustername.  Then when adding an OCUM poller it will be able to find the sites for each cDOT system.

An example of pollers:

[sdt-cdot1]
hostname = 10.64.28.242
site = nl

[sdt-um]
hostname = 10.64.28.77
site = uk
host_type = OCUM
data_update_freq = 900
normalized_xfer = gb_per_sec

So the first poller section is named after the clustername (from 'cluster identity show', or also default prompt from cDOT CLI) and has the IP address and site listed.  The second poller section is for OCUM at the IP address and site (for poller metrics only) listed.  It will then discover all clusters known to OCUM and lookup for a poller section to get the site for those capacity metrics and submit them.  So in this case the cluster sdt-cdot1 capacity metrics will be loaded under site of nl, and metadata metrics for the poller itself will be loaded under site of uk.

I hope this makes sense.  You aren't the first person to have an issue configuring this OCUM integration so I think I need to update the docs to explain it better.

Cheers,
Chris Madden

Storage Architect, NetApp EMEA (and author of Harvest)

Blog: It all begins with data

[SrikanthReddy]

if we have one OCUM monitoring clusters in different locations

how to define config for multiple site clusters  with one OCUM at one site ? 

Adding Chris reply to this 

madden

Capacity metrics for a cluster are submitted using the site from the cluster’s own poller entry.  So the capacity data is in the same site as the performance data because it learns site from the cluster’s poller definition. The site in the OCUM poller is only for metrics from the poller itself like api_time, # metrics submitted, etc.

J_Curl

The "site" value ties OCUM to the cluster. 

[madden]

Hi,

Here could be an example:

[uk-cluster1]
hostname = 10.64.28.242
site = uk

[nl-cluster1]
hostname = 10.64.28.243
site = nl

[jp-unifiedmanager1]
hostname = 10.64.28.77
site = jp
host_type = OCUM
data_update_freq = 900
normalized_xfer = gb_per_sec

1. Harvest will collect capacity info for all clusters defined in jp-unifiedmanager1.
2. Capacity metrics for uk-cluster1 will be submitted under site uk.
3. Capacity metrics for nl-cluster1 will be submitted under site nl.
4. Poller metrics for jp-unifiedmanager1 (like how many metrics it posted, how long it took the UM API to respond, etc) will be submitted under site jp.
5. Any other clusters known to jp-unifiedmanager1 but that don't have a poller entry will not submit metrics and will log a message like: "[2015-10-02 10:59:08] [WARNING] [sysinfo] Discovered [CDOT-2] on OCUM server with no matching conf section; to collect this cluster please add a section".  If you want capacity metrics for CDOT-2 you need to add a poller entry so that the site can be learned.

Hope that helps!

Cheers,
Chris Madden

Storage Architect, NetApp EMEA (and author of Harvest)

Blog: It all begins with data

[mlinzer]

If that's the case, why does my Graphite server only show one site under the "capacity" metrics in the tree?

see attached screenshot. I would expect to see both sites listed under "capacity" the way they both appear under "perf"

Thanks,

Moshe

[madden]

Hi Moshe,

Correct, you should see two sites under netapp.capacity.  Did you check the oncommand logfile in /opt/netapp-harvest/log to get a hint what could be wrong?  

My guess is one of:

(a) mhnap-cls is not added to the oncommand server

(b) it has a different name on the oncommand server

(c) you need to do a F5 refresh in the web browser because the metrics arrived after you loaded the page

(d) you ran out of diskspace and those metrics files can't be created

Cheers,
Chris Madden

Storage Architect, NetApp EMEA (and author of Harvest)

Blog: It all begins with data

[mlinzer · Accepted solution]

Thanks Chris, checking the logs provided the answer. I had made a change to the mhnap-cls entry and never restarted the OCUM poller! Once I restarted it it picked up both entries.

However, I noticed another error in the mhnap-cls log:

[WARNING] [lun] update of data cache failed with reason: For lun object, no instances were found to match the given query.

This cluster has no luns. is that why i am getting these errors? they appear every minute! any way to stop them?

Moshe