I'm not quite sure I understand the question. Are you trying to connect more than 255 LUNs to a single ESX Cluster? iGroups will not allow you to do this. Or are you trying to connect more than 255 LUNs to MULTIPLE ESX clusters?
You can only even have 255 LUN mounted to any given ESX server. Thats an ESX limit. However you really should have all LUNs connected to all ESX servers in a cluster (VMware best practice) otherwise it can become very confusing to try to move VMs around using VMotion as you will have hosts certain VMs can't get to.
I think you have 2 options.
First, use fewer but larger LUNs. Why so many LUNs? With VMFS 5 you can have very large LUNS with hundreds of VMs with no performance loss, so why not reduce the number of LUNs. Besides you will find host reboots to be very slow with 255 LUNs, long HBA rescans.
Or you could break your 11 hosts into multiple smaller clusters then zone the LUNs as you described. Not really any point in having a large Cluster if all the LUNs are not presented to all hosts.
We're having similar concerns with the number of LUN's per ESX cluster. To answer the question about the number of LUN's, one case in point is with SAP (Windows). In order to meet best practice (SAP and NetApp's for SM SAP) we need 7 RDM LUN's per SAP instance (sapdata1-4, origlogA-B, oraarch). We've moved as much as we can onto vmdk's, but 7 seems to be the least we can have in our environment. As a result we are limited to around 37 SAP instances per cluster, which is not many when you account for dev/test and clones used for verifications, etc.
The 256 LUN limit is one imposed by VMware and I'm not sure if they plan on increasing it soon. You should relay this back to your VMware account teams.
One other thing you could do is leverage NFS for your general purpose VM datastores (thus saving you a few LUNs) or map some of the LUNs directly to the VMs using ISCSI again saving some of the 256 LUNs the ESX cluster can see. You of course will have to see if these would work in your environments but the 256 is a hard limit that we can't effect.