nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Grafana tool isnt showing any of the graphs and data

CSS

Grafana tool isnt showing any of the graphs and data , upon checking my grafana debian server server logs i found the below error. Can some one let me know how to resolve this issue.???

“[2019-07-24 12:53:46] [WARNING] [workload_volume] update of data cache failed with reason: Aggregated instances requested for the workload_volume object exceeds the data capacity of the performance subsystem, because it includes 59088 constituent instances. With the current counter set, use the -node, -vserver, or -filter flags to include at most 40206 constituent instances in order to stay within the data capacity. Alternatively, requesting fewer counters will also reduce the required data and may allow more instances to be requested.

[2019-07-24 12:53:46] [WARNING] [workload_volume] data-list update failed.” Hence, you want to reduce the instances or split the instances to make the balancing workload on Grafana, so that the ONTAP would accept the request makes by the Grafana tool.

Find more posts tagged with

Active IQ Unified Manager

Accepted answers

vachagan_gratian

Thanks for the follow-up.

Would you like to set up a call somewhere next week? I really would like to get to the bottom of this case, but it's hard to reproduce in my own environment.

All comments

vachagan_gratian

Setting a custom batchsize for workload_volume might help.

- Open the template that your poller is using (you can find this in one of first log messages printed when your poller starts). E.g., if the template is [cdot-9.3.0.conf], open /opt/netapp-harvest/template/default/cdot-9.3.0.conf in a text editor.

- Find the section workload_volume, which would look something like this

    'workload_volume' =>
            {
                counter_list     => [ qw(instance_name instance_uuid
                                    ops read_ops write_ops
                                    total_data read_data write_data
                                    latency read_latency write_latency
                                    read_io_type sequential_reads sequential_writes
                                    ) ],
                graphite_leaf    => 'svm.{vserver}.workload.{\'policy-group\'}.{volume}.{qtree}.{lun}.{file}',
                plugin           => 'cdot-workload',
                plugin_leaf  => [ 'svm.{vserver}.qos_policy.{qos_policy}', 'svm.{vserver}.vol.{volume}.lun.{lun}',
                                    'svm.{vserver}.vol.{volume}.file.{file}', 'svm.{vserver}.vol.{volume}',
                                    'svm.{vserver}'
                                    ],
                plugin_options   => {'policy-group' => 1, 'workload' => 1},
                enabled          => '1'
            },

- Add the parameter batch_size => 250, on the top, save and restart Harvest.

    'workload_volume' =>
            {
                batch_size => 250,
                counter_list     => [ qw(instance_name instance_uuid ....
                ....

If this doesn't help, let us know!

CSS

@vachagan_gratian Thanks alot for taking your time and replying my query.

As you said I have changed as per the example show below. But still the same problem exits .

cat /opt/netapp-harvest/template/default/cdot-9.3.0.conf

'workload_volume' =>

                        {
                                batch_size => 250,
                                counter_list     => [ qw(instance_name instance_uuid
                                                                        ops read_ops write_ops
                                                                        total_data read_data write_data
                                                                        latency read_latency write_latency
                                                                        read_io_type sequential_reads sequential_writes
                                                                        ) ],
                                graphite_leaf    => 'svm.{vserver}.workload.{\'policy-group\'}.{volume}.{qtree}.{lun}.{file}',
                                plugin           => 'cdot-workload',
                                plugin_leaf      => [ 'svm.{vserver}.qos_policy.{qos_policy}', 'svm.{vserver}.vol.{volume}.lun.{lun}',
                                                                        'svm.{vserver}.vol.{volume}.file.{file}', 'svm.{vserver}.vol.{volume}',
                                                                        'svm.{vserver}'
                                                                        ],
                                plugin_options   => {'policy-group' => 1, 'workload' => 1},
                                enabled          => '1'
                        },

______________________

Files:

root@fgprd-oncommand-graphite-app003:/opt/netapp-harvest/template/default# ls -lrt
total 640
-rw-r--r-- 1 root root 2280 Jan 17 2018 ocum-6.4.0.conf
-rw-r--r-- 1 root root 2251 Jan 17 2018 ocum-6.3.0.conf
-rw-r--r-- 1 root root 2251 Jan 17 2018 ocum-6.2.0.conf
-rw-r--r-- 1 root root 2251 Jan 17 2018 ocum-6.1.0.conf
-rw-r--r-- 1 root root 48549 Jan 17 2018 cdot-9.0.0.conf
-rw-r--r-- 1 root root 47514 Jan 17 2018 cdot-8.3.2.conf
-rw-r--r-- 1 root root 47529 Jan 17 2018 cdot-8.3.0.conf
-rw-r--r-- 1 root root 45162 Jan 17 2018 cdot-8.2.4.conf
-rw-r--r-- 1 root root 43908 Jan 17 2018 cdot-8.2.0.conf
-rw-r--r-- 1 root root 15851 Jan 17 2018 cdot-8.1.0.conf
-rw-r--r-- 1 root root 9910 Jan 17 2018 7dot-8.2.0.conf
-rw-r--r-- 1 root root 9913 Jan 17 2018 7dot-8.1.0.conf
-rw-r--r-- 1 root root 8912 Jan 17 2018 7dot-8.0.0.conf
-rw-r--r-- 1 root root 8545 Jan 17 2018 7dot-7.3.0.conf
-rw-r--r-- 1 root root 49686 Feb 16 2018 cdot-9.1.0.conf
-rw-r--r-- 1 root root 51058 Feb 16 2018 cdot-9.2.0.conf
-rw-r--r-- 1 root root 2280 Feb 16 2018 ocum-7.0.0.conf
-rw-r--r-- 1 root root 2280 Apr 27 2018 ocum-7.1.0.conf
-rw-r--r-- 1 root root 2280 Apr 27 2018 ocum-7.2.0.conf
-rw-r--r-- 1 root root 51058 Jan 7 2019 cdot-9.5.0.conf
-rw-r--r-- 1 root root 51058 Jan 7 2019 cdot-9.4.0.conf
-rw-r--r-- 1 root root 51058 Aug 7 06:16 cdot-9.3.0.bck_aug
-rw-r--r-- 1 root root 51082 Aug 7 06:27 cdot-9.3.0.conf

________________

restarted harvest service.

Recent logs :

[2019-08-07 07:00:54] [WARNING] [workload_volume] data-list update failed.
[2019-08-07 07:06:19] [WARNING] [workload_volume] update of data cache failed with reason: Aggregated instances requested for the workload_volume object exceeds the data capacity of the performance subsystem, because it includes 58248 constituent instances. With the current counter set, use the -node, -vserver, or -filter flags to include at most 40206 constituent instances in order to stay within the data capacity. Alternatively, requesting fewer counters will also reduce the required data and may allow more instances to be requested.

CSS

Can you please help here ??

CSS

This is how my grafana looks now without any data. even after setting batch size to 250.

grafana_error.PNG

vachagan_gratian

Thanks for the follow-up.

Would you like to set up a call somewhere next week? I really would like to get to the bottom of this case, but it's hard to reproduce in my own environment.