Subscribe

instructions for alignment with Linux and LVM

Does anyone by any chance have a nice "cookbook" solution for handling existing Linux VM's using LVM or even new Linux VM's?

I've gone through TR-3749 as well as the VMware alignment document and while I'm comfortable enough with the fdisk steps they are fairly cumbersome from a customer perspective (whether for fixing Linux alignment when can't use mbralign or just for Linux alignment in general).

I'm thinking of working something up but am just pressed for time right now so thought I'd put this out there....

Thanks in advance for any replies.

Re: instructions for alignment with Linux and LVM

And....I figured I'd put this in the public forum just as figured it would be of general interest.

Re: instructions for alignment with Linux and LVM

In my personal opinion, LVM doesn't provide much additional benefit in virtual environments (or on NetApp luns).  I suggest new machines be created with one vmdk file per filesystem.  The only vmdk file in a linux vm that needs a Master Boot Record (MBR) is the boot drive (often mounted as /boot).  The remaining drives do NOT need partitions at all (so no need to use fdisk).  The filesystem can be created directly on the device.  This results in an aligned filesystem.  The filesystem can also be grown as needed, by growing the vmdk file (no need to mess with partition changes) and then growing the guest filesystem.  This basically replaces the need for a volume manager. 

Again, this is my personal opinion, but here is what I tend to do:
boot.vmdk (usually 256MB, a single partition starting at 64 sectors)
swap.vmdk (size depends on the memory in vm, no partition table)
root.vmdk (size depends on distribution, no partition table)
var.vmdk (size depends on applications hosted, no partition table)

Hope this helps.

Re: instructions for alignment with Linux and LVM

So, I am extremely late coming back here....I do appreciate the input however.

The school I came from was basically...

/boot = 100 MB or so

LVM = rest of the space as resizing LV's inside LVM is just very easy

I must admit that dropping LVM for more vmdk's does make a good bit of sense....just push everything into an individual vmdk that you would have had as an individual LVM LV.

And...I hate to ask the obvious....but have you ever seen any issues with dropping LVM? (I'm just so comfortable with it personally that it still just seems really odd to drop it.)

Thanks again.

Re: instructions for alignment with Linux and LVM

Let me start by saying I really like LVM when dealing with physical disk devices.  I was a Unix/Linux SysAdmin in a former life and took advantage of logical volume managers whenever working with physical disk or legacy disk arrays. 

The features I used (depending on the lvm) were;
  • resize logical volume (grow only filesystems in some cases)
  • duplicate logical volume (mirror split for backups, etc)
  • redundancy (RAID protection)
  • snapshot (backup)
The vmdk file per guest filesystem model, when used on NetApp Storage, provides the features tended to use.  I might be loosing the ability to shrink (there are workarounds), but ext3 doesn't shrink anyway (again there are workarounds).  Other than that, I have each of these (and more) using this model. 
I wrote a short blog post on the topic this summer.
I'd be interested in hearing about any other features that may be lost with using this model.

Re: instructions for alignment with Linux and LVM

forgette wrote:

I'd be interested in hearing about any other features that may be lost with using this model.

Host based mirroring between multiple storage arrays. Usually this is done to build disaster recovery solution but can also be to increase resiliency of local config (e.g. there are known low-end raids that won't support online firmware upgrade). Since ESX does not offer any form of hypervisor based  disk mirroring, it has to be done inside VM => volume manager.

Re: instructions for alignment with Linux and LVM

Its probably bad sport to answer my own challenge - "I'd be interested in hearing about any other features that may be lost with using this model.", but here goes anyway...

I've spoken with a few customers who work in environments where the linux administrator does not have rights to create and resize the virtual hard drives.  This obviously shoots my recommendation of 1 filesystem per vmdk.  At this point I'm really glad I said I like LVM in the previous post.  ;-)

In this case, I think Andrew's recommendation is spot on.  The key is that the LVM needs to be properly aligned (each physical volume in LVM2 terms)!

So, Andrew's formula might look like this:
disk.vmdk (two partitions with starting offsets (in sectors) divisible by 8)

For simplicity sake, one might consider the following:
boot.vmdk (usually 256MB, a single partition starting at 64 sectors)
lvm.vmdk (size depends on applications hosted, no partition table)

Please keep a few other things in mind:
1.) Any additional vmdk that you add to the LVM must be properly aligned (simply use the whole device = aligned).
2.) Some utilities may not work with guest filesystems under LVM control.
Hope this helps,
-Eric

Re: instructions for alignment with Linux and LVM

Jon Benedict recently posted this document which details how to create an aligned linux VM using kickstart.

http://communities.netapp.com/docs/DOC-8107

Re: instructions for alignment with Linux and LVM

Hi.

For some reasone the access to this page has been limited. We were able to access it for a while but now we are greeted with "unauthorized" message.

Re: instructions for alignment with Linux and LVM

I'd also like to have access to this document restored...it looked useful but I had not gotten a chance to chare it with our Linux systems team.

Thanks!

Dan