_Missing space in Volume

[karel_heijstek]

Hello,

I've a strange problem. Altough, to me it's strange.

One of our customers has a FAS2220. On it a employee created a Vol with a Total Cap. of 2.65TB.

It was used as a NFS store for VMWare to place VM's on it. At this moment there's (almost) nothing stored on it.

Snapshotting is off. There is no fractional reserve.

Still the filers reports this:

        VOLUME                                        AVAILABLE

     Total Data Space:     2.65 TB      Data Space:            1011.24 GB

     Snapshot reserve:     0 Byte        Snapshot Reserve:    0 Byte

      USED

     Data Space:                       5.42 GB

     Snapshot Copies Space:     0 Byte 

                    Total:                   5.42 GB

So somehow i'm losing 1.6 TB!

Does somebody where can it be or how I can trace it?

Regards,

Karel

[VIRTUALGEEK2]

what reservation have you got for snapshots?

what raid type are you using?

how large is your raid group?

[aborzenkov]

You missing space where? In aggregate? Please, please show output of "df -Ah".

[karel_heijstek]

Hi Virtualgeek2 and aborzenkov!

@Virtualgeek:

     I have no reservation for snapshots on that volume.

     Raid type:  RAID_DP

     I'm not sure what you mean with your third question but we have 2 aggregates. The aggr on which that vol resides has a Total space of 3.85TB from which 2.84TB is used.

@aborzenkov:

     I'm missing space in that particular volume. It has a total space of  265 TB and about 1 TB is available. That means that 1.6 TB is in use, but I have no idea where or what.

     df -AH ouput:

[aborzenkov]

OK, then please paste output of “df -h volume”, “df -hr volume” and “df -hS volulme for the volume where space is missing. Please, paste text in message, not link images.

[karel_heijstek]

OK. I will paste text. Sorry for linking.

Here is the output:

BJZRNAS01> df -h /vol/bjzr_vms

Filesystem               total       used      avail capacity  Mounted on

/vol/bjzr_vms/          2713GB     5177MB     1030GB      62%  /vol/bjzr_vms/

/vol/bjzr_vms/.snapshot        0TB        0TB        0TB     ---%  /vol/bjzr_vms/.snapshot

BJZRNAS01> df -hr /vol/bjzr_vms

Filesystem               total       used      avail   reserved  Mounted on

/vol/bjzr_vms/          2713GB     5177MB     1030GB        0MB  /vol/bjzr_vms/

/vol/bjzr_vms/.snapshot        0TB        0TB        0TB        0TB  /vol/bjzr_vms/.snapshot

BJZRNAS01> df -hS /vol/bjzr_vms

Filesystem                used       total-saved    %total-saved    deduplicated    %deduplicated    compressed    %compressed

/vol/bjzr_vms/          5177MB            2036KB              0%          2036KB               0%           0KB             0%

[VIRTUALGEEK2]

can you please do please do a "vol show" and paste the results for that volume?

[aborzenkov]

Which Data ONTAP verion?

[karel_heijstek]

NetApp Release 8.1.4 7-Mode

[karel_heijstek]

Show is no recognized. Probably you mean vol status?

this is the output:

BJZRNAS01> vol status /vol/bjzr_vms

         Volume State           Status            Options

       bjzr_vms online          raid_dp, flex     nosnap=on, create_ucode=on, convert_ucode=on,

                                sis               guarantee=none, fractional_reserve=0

                                64-bit

                         Volume UUID: 85859b3c-4aa8-11e2-a86d-123478563412

                Containing aggregate: 'aggr0'

[aborzenkov · Accepted solution]

You may be hit by deduplication bug; see https://kb.netapp.com/support/index?page=content&id=7010056&locale=en_US for procedure to check metadata consumption.

[karel_heijstek]

hi aborzenkov,

I will have a look at it! Sounds interesting!

Thanks!

[karel_heijstek]

Hi aborzenkov,

Sorry for the late response. I just want to say that you was right! I told my customer what was possibly the reason, but he wants me to verify it by NetAPP. And guess what? They came up with the same KB article.

Beside the possiblity to just delete the volume they came with another possibility.

I paste what they wrote so that it may help somebody else out there!

***

The ‘sis start -s <vol> command discussed in the KB is resource-intensive. To avoid the performance hit,

you can just reset the sis process for the volume. That will clear out all metadata and allow the

fingerprint database to start fresh manually. It will not eliminate the deduplication that has already

been performed.

Stop deduplication for the volume if it is active or queued. Afterward, it can be restarted:

     filer> sis stop -a /vol/volume_name

     The sis reset command:

     ---------------------------------------------

     filer> priv set diag

     filer*> sis reset /vol/volume_name

     filer> priv set admin

The ‘sis reset’ commands should run in a matter of seconds, a minute at worst.

*Note*: ‘sis reset’ is interactive and will ask some confirmation questions.

This will delete all the volume’s metadata in the volume and aggregate, but will not affect the

currently deduplicated data. However, it will require a rebuild of the tables from scratch.

*****

Thanks for your help!

Regards,

Karel