Any further details on this from NetApp by any chance? Especially when running VMware environments with dedupe, I find reallocate to be critical....so losing it when running MetroCluster (as I have one customer doing) is a serious issue IMHO.
Re: reallocate on a metrocluster - split the mirror?!
(1) The command reallocate start -f /vol/esxprod2_nfs_sys will indeed give an error if you have snapshots (because of the high rate of changed blocks that might result in filling up your vol) but you can run it per LUN/file like reallocate start -f /vol/esxprod2_nfs_sys/data.vmdk. Maybe give it a try on some VMs that you think are most impacted.
(2) If you have a metrocluster you can enable reads to hit both plexes as by default its set to local plex only. If you have a long distance between sites, or know you'll saturate your ISLs in a fabric MC then no solution but for many situations it allows you to double your read IOPs at no cost. The behavour is configured by a system-wide option, can be done online, and takes effect immediately:
raid.mirror_read_plex_pref Specifies the plex preference when reading from a mirrored traditional volume or aggregate on a metrocluster-configured system. There are three possible values -- `local' indicates that all reads are handled by the local plex (plex consisting of disks from Pool0), `remote' indicates that all reads are handled by the remote plex (plex consisting of disks from Pool1), and `alternate' indicates that the handling of read requests is shared between the two plexes. This option is ignored if the system is not in a metrocluster configuration, i.e., cluster_remote is not licensed. The option setting applies to all traditional volumes and aggregates on the filer.
Physical reallocation of mirrored aggregates is now supported. I found the following in the release notes of 8.1.1 (SE told me to have a look at it):
Reallocation scans, read reallocation, and extents with FlexVol volumes contained in mirrored aggregates
In prior releases, you could enable reallocation scans, read reallocation, and extents on FlexVol volumes contained in mirrored aggregates. However, some command options were not supported with this type of volume.
Starting in Data ONTAP 8.1.1, you can use the following command options to optimize the layout of FlexVol volumes that are contained in mirrored aggregates:
You can use the -p option when you run the reallocate start command. The -p option (also called physical reallocation) reallocates user data on the physical blocks in the aggregate while preserving the logical block locationswithin a FlexVol volume.
You can set the read_realloc option to space_optimized when you run the vol options command. space_optimized conserves space but results in degraded Snapshot read performance.
You can set the extent option to space_optimized when you run the vol options command. space_optimized conserves space but results in degraded Snapshot read performance.