Discrepancy between windows explorer total file size and df output on filer

chris · ‎2010-08-25

I'm trying to figure out how to account for the difference in file sizes reported when viewing the total size of all the files in a volume via cifs with windows explorer and via df from the filer. If I select all files and folders in the root of a cifs share, right click, and hit properties the size reported by windows is 292G while the used reported via df is 277G. I can't for the life of me figure out how to account for this discrepancy.

The volume is exported via cifs and nfs, guarantee is set to volume and there are no snapshots of this volume. The volume contains nothing but files and folders created via cifs and nfs clients, there are no luns.

Thanks,

Chris

eric_sherrill · ‎2010-08-25

I presume also, no dedupe (sis off), and no cifs file folding is enabled? Then probably just the good old confusing NetApp "GB" (1000 MiB or 1000 x 1024^2 bytes) vs. binary GiB (1024^3 bytes - typical SAN vendor & MicroSoft usage) vs. base-10 GB (1000^3 bytes - typical disk labeled capacity and actually the SI "standard") discrepancy.

See also http://en.wikipedia.org/wiki/Gigabyte

chris · ‎2010-08-26

Hi Eric,

Right, no dedup, no cifs folding. The df man page on the filer states "Unit values are based on powers of two. For example, one megabyte is equal to 1,048,576 bytes." So it looks like it's not GB vs GiB.

On the netapp with df I get 290673272KiB or 297649430528 bytes.

Windows Explorer (using the technique I mentioned earlier) displays 314462265344 bytes or 292GiB. Mapping that share to a drive and checking the drive properties reports values inline with the output of df on the filer: 277 GiB used.

Mounting the share via cifs from a linux box and running df produces output identical to the output of the df command on the filer. What's interesting is that if I run du -hs at the root of the mounted volume I get back 275G. The man page for du states that it "estimates" file space disk usage so I'm assuming that accounts for the 2G discrepancy there. Running du -hs --apparent-size instead returns 293G which is inline with what windows reports 314462265344 bytes = 292.86580658GiB. The man page for du says about the apparent size flag:

              print apparent sizes, rather than disk usage; although the apparent size is usually smaller,
              it may be larger due to holes in (‘sparse’) files, internal fragmentation, indirect blocks,
              and the like

So I guess selecting all the files and folders in the share, right clicking, and doing a properties in windows is roughly analgous to du -s --apparent-size and the discrepancy I'm seeing is caused by "holes in (‘sparse’) files, internal fragmentation, indirect blocks, and the like"

Thanks,

Chris

eric_sherrill · ‎2010-09-30

The NetApp KB and MB are really KiB and MiB, but the NetApp GB is 1000 MiB. Presumably the TB is 1000 of these unique NetApp "GB". I am basing this on the V-Series documentation:

"The Data ONTAP definition of a GB is that one GB is equal to 1000 x 1024 x 1024 bytes." See e.g. http://now.netapp.com/NOW/knowledge/docs/ontap/rel7261_vs/html/v-series/ig_hit/intro5.htm