Lun types for ESX SAN Boot

[REGIS_GARRUCHET]

Hello,

it may seems a silly question, but I've been looking for an answer for some times and didn't find any document that answer to this question. I see also on several forums that I'm not the only one asking for this question.

What is the favourite lun type for San Boot of ESX/ESXi server (in my case version 5, but I guess any clue for other version would be great).

My guess is that the linux lun type would be the good one (since it's not datastore luns) but i can't find anything in NetApp docs, TR, kb that confirms this.

Another question (depending on the answer for the previous one)  is:

My customer already installed ESX on luns with vmware type. Should he reinstall it or can it run like that without a problem ?

Thank you.

Regis

[superderivatives]

Hi,

the lun type for esx BootFromSAN should be VMware.

good luck,

Ariel

[dburkland]

I used "vmware" as the lun type and the filers have been reporting that the LUNs are misaligned. Has anybody else ran into this?

[maske]

NetApp best practices state that the swap files for VMs should always be on local disk to avoid alignment issues.  Have you spun up any VMs yet?

[superderivatives]

yes. luns created via oncommand or command line will be misaligned.

best is to create them via snapdrive.

good luck

Ariel

[keitha]

Yes, this is known. One of the partitions that ESX creates is misaligned. Since it is a single partition within the LUN it is impossible to correct. However after boot ESX rarely ever touches these LUNs so I would not worry about then generating Misalignment.

Keith

[REGIS_GARRUCHET]

Yes, you're right. That's why I ask.

And the following thread from the NetApp support Community states that Linux would be the good choice for ESX Boot from SAN luns :

https://forums.netapp.com/message/139825#139825

What I understand from the readings I had since yesterday is:

- vmware lun type is for datastores and when applied to system boot luns, there is misalignment. But it can be choosen for boot luns.

- linux lun type would fit well but it's not stated anywhere in the docs.

- Anyway, misalignment on boot luns should not be an issue since there is no heavy load on these luns.

Am I right ?

Regis

[maske]

Misalignment can affect the performance of all of the disks in the aggregate since the misaligned I/O is across all of the disks in the aggregate.  You can see how the disks are performing by running:

statit -b

(wait 10 seconds)

statit -e

You may have to run 'priv set advanced' or 'priv set diag' to see these commands, don't forget to run 'priv set' to end advanced or diag mode.

[dburkland]

Regis,

That is what I heard from several NetApp SEs however I could never find anything relevant in the available documentation. Technically there is not a lot of IO going to these LUNs but I can see how it could impact overall performance of the aggregate.

Dan

[maske]

The disk reads have to work twice as hard.  ONTAP reads in 4k blocks so if some if the 4k blocks are misaligned you'll have to read 4k blocks twice for every one 4k block read doubling the work done on disk because the 4k blocks are written, for example, 2k in the first 4k block and 2k in the next 4k block.  Depending on the misalignment this would impact every disk with misaligned data on it in the aggregate.

[REGIS_GARRUCHET]

When we do a boot from SAN for Linux, it's simple, we choose Linux. When we do a boot from SAN for Windows, it simple as well, we choose Windows (Windows_2008 or GPT depending on the version)

When we do a boot from SAN for ESX, we could choose:

- Linux: because it's a Linux-like OS with Linux-like partitions, and it seems to fit well.

- VMWARE: because it's VMWARE. But there is an issue doing this: VMWARE lun type has a good alignment for VMFS datastores and I don't think system luns use the same alignement than datastores. And Dan and Keitha confim there is misalignment seen when using vmware lun type.

The question is which choice is better and an official statement somewhere in NetApp docs would really be great.

off topic:

- About the impact of alignment on system luns, I agree with Keitha: there is nearly no I/Os on system luns for ESX so there should be little to no impact on the aggregate. If disks have to work twice as hard for near-zero I/Os, there is no real problem.

- Doug, you are right, misalignment on VMware can be a real pain, but it usually occurs because of misaligned vmdks on datastores (very often caused by a bad P2V conversion) or by a bad lun type for RDM luns. It's never an issue coming from the system luns.

That why every administrator should pay attention to vmdk alignement and use mbralign tool when necessary or, if VMs cannot be stopped, use the new feature from the VSC to create a misaligned datastore, so the misaligned vmdks become aligned with ONTAP luns: the same way system luns don't produce a lot of I/Os, datastore don't produce a lot of I/Os. Most of I/Os come from vmdk, that why these have to be aligned.

- And finally when there is such misalignment issue, it only has a real impact when the aggregate is already under heavy load, and Cache and NVRAM can't hide the problem anymore.

(BTW, be careful in reading ONTAP CLI report for alignments: partial reads on RDM Log luns don't mean there is alignment issue for example, but they are often seen as unaligned ).

[keitha]

Great thread guys. There certainly is alot of confusion around alignment and VMware. There are a couple good whitepapers out there, I'll try to track them down and post them back here but in the mean time here is a quick explanation that hopefully helps.

First it is important to understand what setting the LUN type on NetApp does. It adjusts the starting offset of the LUN on NetApp so that the guest filesystem that is installed into the LUN correctly aligns with the underlying WAFL blocks. Not every filesystem has a different offset though. For example both VMFS and Linux don't require any offset at all. So this means when you create a LUN of type VMware we don't offset the LUN at all and when you install VMFS into that LUN the VMFS blocks align with the WAFL blocks. Same with Linux and ESX 3.

So far so good?

In this thread we were talking about ESX Boot Luns. Now the confusion is this, should be pick type VMware since this is ESX or Linux since the guest OS is Linux? Well, technically it should be type Linux since you will be installing EXT3 in the LUN not VMFS. ( I guess we could list the filesystems instead of the OS types and that might help the confusion) however since both Type VMware and Type Linux have an offset of 0 then it doesn't really matter.

Now we also make an assumption that you will either install a single partition/filesystem into the LUN or insure that each partition will have the same offset and alignment. Unfortunately with ESX this is not the case. The ESX boot image has several partitons a few of which have a different starting offset. This means we can't offset the LUN on the NetApp and get all the partitions to line up correctly.

Again though, not to worry, the ESX boot image does so little IO that is isn't worth worrying about.   ONTAP is able to handle a reasonable amount of misaligned IO with minimal performance impact and this would be the case for the ESX boot LUNS.

Now I won't go too deep into the VM alignment problem since this thread was about the ESX boot LUNS not the VM datastores but someone brought it up so here is the important difference. The challenge with VM datastore LUNs is the fact you are dealing with 2 file systems. Yes you select type VMware which is insure the VMFS file system aligns with the NetApp WAFL layout but then on top of the VMFS you install NTFS or EXT3 which may or may not align with everything below it...Thus misalignment.

I hope this helps