Options
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Does anyone by any chance have a nice "cookbook" solution for handling existing Linux VM's using LVM or even new Linux VM's?
I've gone through TR-3749 as well as the VMware alignment document and while I'm comfortable enough with the fdisk steps they are fairly cumbersome from a customer perspective (whether for fixing Linux alignment when can't use mbralign or just for Linux alignment in general).
I'm thinking of working something up but am just pressed for time right now so thought I'd put this out there....
Thanks in advance for any replies.
14 REPLIES 14
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
And....I figured I'd put this in the public forum just as figured it would be of general interest.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
In my personal opinion, LVM doesn't provide much additional benefit in virtual environments (or on NetApp luns). I suggest new machines be created with one vmdk file per filesystem. The only vmdk file in a linux vm that needs a Master Boot Record (MBR) is the boot drive (often mounted as /boot). The remaining drives do NOT need partitions at all (so no need to use fdisk). The filesystem can be created directly on the device. This results in an aligned filesystem. The filesystem can also be grown as needed, by growing the vmdk file (no need to mess with partition changes) and then growing the guest filesystem. This basically replaces the need for a volume manager.
Again, this is my personal opinion, but here is what I tend to do:
boot.vmdk (usually 256MB, a single partition starting at 64 sectors)
swap.vmdk (size depends on the memory in vm, no partition table)
root.vmdk (size depends on distribution, no partition table)
var.vmdk (size depends on applications hosted, no partition table)
Hope this helps.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
So, I am extremely late coming back here....I do appreciate the input however.
The school I came from was basically...
/boot = 100 MB or so
LVM = rest of the space as resizing LV's inside LVM is just very easy
I must admit that dropping LVM for more vmdk's does make a good bit of sense....just push everything into an individual vmdk that you would have had as an individual LVM LV.
And...I hate to ask the obvious....but have you ever seen any issues with dropping LVM? (I'm just so comfortable with it personally that it still just seems really odd to drop it.)
Thanks again.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Let me start by saying I really like LVM when dealing with physical disk devices. I was a Unix/Linux SysAdmin in a former life and took advantage of logical volume managers whenever working with physical disk or legacy disk arrays.
The features I used (depending on the lvm) were;
- resize logical volume (grow only filesystems in some cases)
- duplicate logical volume (mirror split for backups, etc)
- redundancy (RAID protection)
- snapshot (backup)
The vmdk file per guest filesystem model, when used on NetApp Storage, provides the features tended to use. I might be loosing the ability to shrink (there are workarounds), but ext3 doesn't shrink anyway (again there are workarounds). Other than that, I have each of these (and more) using this model.
I wrote a short blog post on the topic this summer.
I'd be interested in hearing about any other features that may be lost with using this model.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
forgette wrote:
I'd be interested in hearing about any other features that may be lost with using this model.
Host based mirroring between multiple storage arrays. Usually this is done to build disaster recovery solution but can also be to increase resiliency of local config (e.g. there are known low-end raids that won't support online firmware upgrade). Since ESX does not offer any form of hypervisor based disk mirroring, it has to be done inside VM => volume manager.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Its probably bad sport to answer my own challenge - "I'd be interested in hearing about any other features that may be lost with using this model.", but here goes anyway...
I've spoken with a few customers who work in environments where the linux administrator does not have rights to create and resize the virtual hard drives. This obviously shoots my recommendation of 1 filesystem per vmdk. At this point I'm really glad I said I like LVM in the previous post. 😉
In this case, I think Andrew's recommendation is spot on. The key is that the LVM needs to be properly aligned (each physical volume in LVM2 terms)!
So, Andrew's formula might look like this:
disk.vmdk (two partitions with starting offsets (in sectors) divisible by 8)
For simplicity sake, one might consider the following:
boot.vmdk (usually 256MB, a single partition starting at 64 sectors)
lvm.vmdk (size depends on applications hosted, no partition table)
Please keep a few other things in mind:
1.) Any additional vmdk that you add to the LVM must be properly aligned (simply use the whole device = aligned).
2.) Some utilities may not work with guest filesystems under LVM control.
Hope this helps,
-Eric
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Jon Benedict recently posted this document which details how to create an aligned linux VM using kickstart.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi.
For some reasone the access to this page has been limited. We were able to access it for a while but now we are greeted with "unauthorized" message.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I'd also like to have access to this document restored...it looked useful but I had not gotten a chance to chare it with our Linux systems team.
Thanks!
Dan
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I can't get to the document anymore either, but there is some good news...
After some searching I found this link:
http://communities.netapp.com/message/39917
The content is also in the latest version of NetApp TR-3747 “Best Practices for File System Alignment in Virtual Environments” as well.
I hope this helps!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Related:
Super Star CSE Seth Forgosh recently posted his notes for dealing with misaligned LVM based disks.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Eric,
In some of the above comments you suggest using a single, non-partitioned, disk for LVM and then lvolumes on top of that. I've been unabel to get kickstart to create such a setup, I can't define a pv partition using the part command to be an entire disk:
part pv.32 --onpart sdb
and if I pre-create the pvolume in %pre, the volgroup command fails. Same if I pre-create the vgroup in %pre, the logvol fails with a "no volume group exists with the name..." error.
The notes you link to don't seem to address this, any thoughts?
A
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Sorry, I don't think kickstart can handle that type config (I'm happy to be wrong ).
Check out section 4.1 of this TR - http://media.netapp.com/documents/tr-3747.pdf where Jon Benedict describes how to create properly aligned partitions in kickstart.
-ef
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi!
I've created this document based on Seth's doc. Maybe it can help you.
https://communities.netapp.com/docs/DOC-12170
Regards,
Pedro Rocha.
