2009-04-17 01:29 AM
I am monitoring a NetApp system and I've found that "allocated space" and "used space" for one aggregate are different. I understand that the "used space" could (or should) be bigger than the allocated, but not the opposite.
The output of the commands are:
> aggr show_space aggr0
Aggregate Allocated Used Avail
Total space 1424727976KB 674527236KB 10953228KB
Snap reserve 75191224KB 15338328KB 59852896KB
WAFL reserve 167091608KB 105020KB 166986588KB
> df -A aggr0
Aggregate kbytes used avail capacity
aggr0 1428633264 1417679976 10953288 99%
aggr0/.snapshot 75191224 15338328 59852896 20%
How can be the allocated space more than the used space ?
2009-04-17 05:36 AM
I am relatively new to aggregates, but I am wondering off the top of my head if you have a issue with snapshots consuming production space.
If you do a df -k or df -g do you see snapshots taking 100% or more?
Something like this example:
/vol/unknownvol01/ 1468006400KB 1039536716KB 428469684KB 71% /vol/unknownvol01/
/vol/unknownvol01/.snapshot 367001600KB 354114616KB 12886984KB 96% /vol/unknownvol01/.snapshot
2009-04-17 05:53 AM
Well, the only snapshot that is over 100% is this one (the rest, both production and snapshot space are under 100%)
/vol/vol2/ 104857600KB 24343704KB 80513896KB 23% /vol/vol2/
snap reserve 26214400KB 30999312KB 0KB 118% /vol/vol2/..
But if it is consuming part of the production space, I still don't understand why the allocated space should be different (I mean, in the allocated space is included the snap and production data, as well as the metadata needed for managing the volume, so if the snap is consuming part of the production space, the allocated space should be the same, with the only different that I won't have all the production space available for data).
2009-04-17 10:45 AM
Found this in a NetApp doc. Can't recall which one, but I think it has your answer.
Total allocated space in the aggregate that is shown by aggr show_space which is the sum of space allocated for all flexible volumes. Used space in the aggregate shown by df -A is slightly larger because it also includes some metadata required to maintain the aggregate
2009-04-19 11:55 PM
Thanks for your effort and interest Anthony.
Yes, that's why I say that I understand that the result of df -A would be greater than the output of aggr show_space, but in my case is the opposite:
df -A reports 1417679976 KB
aggr show_space reports 1424727976 KB
I am trying to find what this happens but I have no clue.
2009-04-21 03:04 PM
This could easily happen if you have thin provisioned the storage anywhere. You can then over allocate as much space as you like!
Check your volumes to see if any are not set to "space reservation = volume" and also check any LUNs to see if they are space reserved.
There is also the unknown in this. To free up data blocks the filer does a scheduled disk scrub, this is done at the plex level. This will systematically go through each block on the system and check for any references to it, if there are none, then it will scrub (format) the block and it will be free again. If your system is particularly busy, or you have deleted a large amount of data, this can lead to quite a large are of space that is unclaimed! This will get freed up over time. In a normal production system you will always find a discrepency though because of this, the filer simply hasn't had chance to go back through and scrub the blocks free just yet.
2009-04-22 01:36 AM
Thanks for your answer Chris, but I still don't understand the problem why the allocated space is greater than the used.
About volumes and LUNs, every volume is set to "space reservation - volume", and I have no LUNs configured. About the disk scrub, that will explain if I would have some lost space or if the allocated space would be much lower than the used space. But the volumes space and the used space are concordant (actually, the addittion of every value of the "df -k" command output is just a bit lower than the used space, but this doesn't include the metadata needed for the volume maintenance, so I see a direct relation between "df -k" and "df -A", but not with "aggr show_space").
If I am not wrong the REAL free space is shown in "df -A", where the addition of used and free space are exactly the usable space for the aggregate.
So my problem is still the difference between the allocated space output for "aggr show_space" and the used space output for "df -A". Can it happen that they look at the blocks usage in some different way?