NaBox C-Mode FPolicy Stats

[darraghos]

Guys, can one configure harvest in NaBox to gather the fpolicy engine stats from a c-mode cluster? I'd like to build a dashboard for this. 

[AlexSun0302]

You need custom by yourself.

[darraghos]

Thanks but where to configure that collection? Happy to build the dashbaords but looking to see where to configure the collection within Harvest on the appliance.

[vachagan_gratian]

Hi @darraghos,

Which stats are you specifically interested in? Right now Harvest doesn't collect fpolicy counters, but those are available (see below, I have a list).

We plan to add an extension module in Harvest 1.5, so you can configure it to collect counters for customized Grafana dashboards. Maybe I can even help you, so you don't have to wait for 1.5, but I don't promise.

The counters available for fpolicy:

                        aborted_requests: Number of screen requests aborted
                           cifs_requests: Number of cifs screen requests sent to fpolicy server
                             event_count: Array of events resulting in notifications
            file_generated_during_outage: Number of outage files generated to store screen requests during network outage.
                           instance_name: Name of the Vserver
                           instance_uuid: ID of the Vserver
                     max_outage_duration: Maximum outage duration at any time.
                max_outstanding_requests: Maximum number of screen requests in process at one time
  max_outstanding_requests_during_outage: Maximum number of screen requests queued at one time during the outage.
                     max_request_latency: Maximum screen requests latency
                            nfs_requests: Number of nfs screen requests sent to fpolicy server
                               node_name: System node name
                               node_uuid: System node id
                    outstanding_requests: Total number of screen requests in process
      outstanding_requests_during_outage: Number of screen requests queued for write to file.
                            process_name: Ontap process that provided this instance
                      processed_requests: Total number of screen requests went through fpolicy processing
        processed_requests_during_outage: Total number of screen requests went through fpolicy resiliency processing.
                    request_latency_hist: Histogram of latency for screen requests
                requests_dispatched_rate: Number of screen requests dispatched per second
          requests_dropped_during_outage: Number of screen requests not stored in file due to resiliency timeout.
                  requests_received_rate: Number of screen requests received per second
     requests_write_failed_during_outage: Number of screen requests not stored in file due to write failures.
    total_requests_dropped_during_outage: Number of screen requests not stored during the network outage.
                    volops_notifications: Number of volume operation notifications sent to fpolicy server

P.S. those are the counters of the object fpolicy, but some related objects are also available, so if you don't find what you need here, we can look in those.

[darraghos]

Thanks! So I should be able to amend the template that defines the collected counters within NABox underlying OS/FileSystem and thus expose these counters to Grafana?

[vachagan_gratian · Accepted solution]

Yes, if the collected counters don't need to be postproccessed (i.e. if the raw counter values are okay).

[naveens17]

Here is custom script I wrote

##
%poller = (
##
## fpolicy counters
##
'fpolicy_policy' =>
{
counter_list => [ qw(instance_name instance_uuid
processed_requests outstanding_requests max_request_latency max_requests_waiting_for_send request_latency_hist)],
graphite_leaf => 'svm.{instance_name}.fpolicy_policy',
enabled => '1'
}

);
##
## Counter definition overrides
##
## Some counter definitions in Data ONTAP are defined incorrectly. This section allows overriding the
## metadata config from Data ONTAP enabling standardized processing and calculation in harvest.
##
%override = (
'fpolicy_policy' =>
{
request_latency_hist => {properties => rate, unit=>microsec}
}
);

Once the data is collected on graphite side, I have made a dashboard on grafan by calling as fpolicy stats and select the correct variables.

[vachagan_gratian]

Great job!

[darraghos]

Finally coming back to this! Thanks so much for the script. From a read on the guide the actual collection templates are perl code so I can create a new template from this code and start collecting?

[bkamil]

---

@naveens17 wrote:

Here is custom script I wrote

 [ ... ]

 Once the data is collected on graphite side, I have made a dashboard on grafan by calling as fpolicy stats and select the correct variables.

---

Thanks a lot!

Would you mind sharing the Grafana dashboard, too?

[darraghos]

I'm going to be that annoying Noob! So, looking at the logs on system I can see that its using the following template for my Cluster  namely /opt/netapp-harvest/template/default/cdot-9.3.0.conf. So, I added the perl lins you kindly posted and restarted the system and have waited several polling intervals but in the graphite browser I canot see these metrics anywhere Might you point me in the right direction here please? 

[darraghos]

Ignore my last post! I was being lazy and should have fully read the admin guide before posting (apologies for being annoying!) . I should have put any custom polling into a new file in the templates folder and modified the poller config file to use this config file along with the default template. I had added this new counter object to the default config but alos just pasted it in thus breaking the config as there can only be one %poller section per config file it seems! That was basic from me.

All working now! Thanks so much for the help guys.