Community

Subscribe
Highlighted

Isci Lun going offline with msg volume space full

have create volume of 96gb and inside it created a lun of size 65gb. 

rsh filer0011 df -h exch_index_1

Filesystem               total       used      avail capacity  Mounted on

/vol/exch_index_1/        96GB       65GB       30GB      68%  /vol/exch_index_1/

/vol/exch_index_1/.snapshot       24GB        0KB       24GB       0%  /vol/exch_index_1/.snapshot

I gues the volume got formatted at windows end so that is why it showing me 65gb used. Please correct me if wrong .

I have set volume option Fractional Reserve to 0  and have the snapshot enabled . Also the volume is getting snapmirror to DR filer.

1) When user deletes the few gb's of data and add fresh data  the all deleted data goes into snapshot and volume gets full and it takes lun offline.

Wed Apr 24 11:33:03 IST [filer0011: scsitarget.lun.noSpace:error]: LUN '/vol/exch_index_1/exch_index_1' has run out of space.
Wed Apr 24 11:33:03 IST [filer0011: lun.offline:warning]: LUN /vol/exch_index_1/exch_index_1 has been taken offline

2) To avoid this I have disabled the snapshot generation and break the snapmirror relation ship with DR volume . Now again when use starts dumping the fresh data it started eating up the snapshot space (snapmirror snapshot started growing ) and once vailable space in volume is eaten up by snapshot the volume gets full and LUN goes offline again.

Below is volume status when only snapmirror snapshot was there and user was dumping fresh data.

rsh filer0011 df -h exch_index_1

Filesystem               total       used      avail capacity  Mounted on

/vol/exch_index_1/        96GB       65GB       30GB      68%  /vol/exch_index_1/

/vol/exch_index_1/.snapshot       24GB       15GB     9133MB      63%  /vol/exch_index_1/.snapshot

rsh filer0011 snap list exch_index_1
Volume exch_index_1
working...

  %/used       %/total  date          name
----------  ----------  ------------  --------
19% (19%)   13% (13%)  Apr 26 10:06  banfpbcpnap0010(0151729288)_exch_index_1R.5 (snapmirror)

To overcome this i have removed snapmirror relationship and delete the snapmirror snapshot as well.

Now its working fine but this is not what i want it.

I want to keep the snapmirror relation on and dont want volume to go offline every now and then when user delete and add more data.

What is causing volume full , why new data is going in snapshot not in actual volume. anything wrong with Fractional Reserve setting ( set to 0 ) . I read full discussion it saying it happens when

Fractional Reserve set to 100%.

Please help.

Re: Isci Lun going offline with msg volume space full

Hi,

you need to look at the "snapshot reserve" for the volume level, from looking above if you take your 96gb volume and then you put a thick provisioned 65gb LUN within that volume. this leaves you 31gb free space in the Volume. this will be used by the snapshots if no reserve is set. from above i can see that you have one snapshot of 15gb which would total 80gb. However check the snapshot reserve and let us know what that is, as that percentage will take the space from the volume total size.

explained by Chris Kranz here - http://communities.netapp.com/groups/chris-kranz-hardware-pro/blog/2009/03/05/fractional-reservation--lun-overwrite

Re: Isci Lun going offline with msg volume space full

fractional_reserve is already set to 0 and snap reserve is 20%. Some one has recommened me that only setting up the fractional_reserve to 0 will not help i need to set volume guarantee=none. I have set that.

Re: Isci Lun going offline with msg volume space full

Hi,

New data doesn't 'go into' the snapshot rather than the 'actual volume'.

The volume is a space which contains blocks. In your case, you have a LUN in there, which contains data presented to a server via iSCSI. Since you have snapshots enabled on this volume, it will periodically snapshot (stop changing blocks which already exist on the volume) the volume, and record any changed blocks in a new area. This isn't literally how it works, but you can think of it like that. This means that any changed blocks count towards the snapshot space, not just deleted blocks. So when your user deletes 20GB of data, then writes 20GB of new data, this creates a 40GB snapshot, not a 20GB snapshot.

Your volume offlines to protect itself, as it has nowhere to write new data.

To prevent this, you have a few options:

Stop creating snapshots (no good if you want to have snapshots)

Increase the overall volume size

Keep fewer snapshots (you can either change the retention schedule, or configure snapshot auto deletion)

Decrease the rate of change in the volume (no good if you want to keep using your volume normally).

Please do a 'snap list' to check if there are any older snapshots consuming space inside your volume.

Also, what is your snapshot schedule?

volume guarantee=none only thin provisions your volume and has no effect on its configured size. It will use less space in the aggregate (maybe), but won't actually be any larger/smaller as far as your LUN is concerned.

I would suggest:

Check the snapshot schedule isn't keeping too many snaps.

Work out how much snapshot space you'll need. Do this by calculating the size of your snapshots multiplied by the number you want to retain. Eg, if your snapshots are 15GB and you want to keep 6 of them, you'll need about 90GB of space to do this.

Then you'll have to size your volume to make sure that you have enough space to contain the LUN, the total snapshot size you want to keep, and then some free space.

The Snapshot reserve is an area for use by snapshots only, however they will use space outside of this reserve as required. Given your LUN can't grow by itself, you shouldn't need to configure this option here, and you can set it later once you know how large your snaps will be. The setting isn't necessary for LUN volumes.

Re: Isci Lun going offline with msg volume space full

i also have the same issue.

i did create a LUN inside a qtree ( enabled quotas ).. no snaps..

as soon as the quota filled up, it took down the LUN

do we have any solutions so that LUN never go into offline mode.

Re: Isci Lun going offline with msg volume space full

Best Practice...

qtree = good

quota = not good

no quota = no offline LUN, easy

Re: Isci Lun going offline with msg volume space full

it did solve the issue..

so whats the difference if we enable quotas, why its taking LUN into offline mode?

we are creating a LUN inside a qtree !!!

can you please share your thoughts what would be the difference when we create a LUN inside a qtree vs LUN inside a qtree which has quotas turned on.

Re: Isci Lun going offline with msg volume space full

Hi,

Thin provision everything (no volume guarantees, no lun space reservation) and stop using snap reserve (set it to 0). Configure SME To not complain about the thin provisioning.

Set up volume auto sizing and warning levels on your aggregates and live a quiet life.

S.

(If it hurts, stop doing it...)

Re: Isci Lun going offline with msg volume space full

thats not the solution i m looking for

Re: Isci Lun going offline with msg volume space full

we are in prob again... even tough the LUN says its full from windows side...

is still accepting to create small files ( 3kb or 2 kb) and we try to open those files, data is corrupted.