Gathering Disk Stats - how?

[chad_w_wilson]

Here is where I am at. My company will not purchase any software to monitor the filer. I can get some stats from snmp, even more from "stats".

However, I am not happy with the data I am able to gather for disk stats, specifically total ops per disk, total ops per raid group, and disk utilization per disk/rg/aggr/volume.

When using sysstat, it is my understanding that the disk utilization displayed is that of this highest disk. This is not useful considering there are multiple raid groups and aggregates.

So trying to use 'stats show disk:*:disk_busy" allows me to get per disk data and do whatever calculations I want, but unfortunately it uses UID as the instance, not disk id.

I can not find a way to use the information from "stats show disk" to determine what aggregate or raid group, or volume the disk belongs to. More so, I can not determine what type of disk it is, SAS, SATA, etc.

The big picture is that I want to gather stats to trend disk i/o performance.

[nitish]

Try statit command to collect data

It's quite useful for analyzing disk/aggregate performance.

disk             ut%  xfers  ureads--chain-usecs writes--chain-usecs cpreads-chain-usecs greads--chain-usecs gwrites-chain-usecs

/aggr0/plex0/rg0:

1a.17              1   2.62    0.35   1.00 26214   2.26   5.62   465   0.01  22.00   182   0.00   ....     .   0.00   ....     .

2a.16              2   2.90    0.34   1.00 38407   2.55   5.21   500   0.01  32.00    94   0.00   ....     .   0.00   ....     .

2a.18              1   1.73    0.61   1.00  8102   1.08  10.67   544   0.04  12.33   514   0.00   ....     .   0.00   ....     .

/aggr2_fc_t1/plex0/rg0:

1c.16              9  27.32    0.30   1.00  7042  11.49  28.05   192  15.52  17.04   172   0.00   ....     .   0.00   ....     .

2c.23              9  27.59    0.30   1.00 26958  11.88  27.19   194  15.41  16.92   199   0.00   ....     .   0.00   ....     .

2c.28             15  43.79   19.02   2.84  2709   7.89  12.03   585  16.88   7.06   385   0.00   ....     .   0.00   ....     .

2c.32             14  41.47   16.16   3.01  2757   8.03  13.10   545  17.28   7.08   342   0.00   ....     .   0.00   ....     .

[chad_w_wilson]

I will have to look at that and see if I can get it to run more quickly than what I am accustomed to it running. THanks.

[aborzenkov]

You can map UUID to pretty disk name(s) using (e.g.) output of "storage show disk -v".

[chad_w_wilson]

Thanks. I used several different tools (stats, aggr, and storage) and a lot of shell utilities to tie them together. It works for now, but I don't like it.

Statit would take to long, as I need to generate this data > 1/min.

[aborzenkov]

Well ... if you are experienced programmer you may find it easier to use Data ONTAP SDK with Perl/Java/PowerShell/C# (and I guess there are are a couple more bindings). This at least avoids need to rely on uncommitted text representation.

Performance counters are also collected by filer as hourly stats and per-second performance archives. They are just XML files after all and so processing could be automated as well. Format is not (publicly) documented, but can be guessed with enough motivation

[chad_w_wilson]

Humm, and where are these performance counters located?

[aborzenkov]

Should be in /etc/log/stats/archive on current versions. But I apologize, I was wrong. While some part of file is indeed XML, actual data is binary. The hour stats are in /etc/log/cm_stats_hourly and are pure text.