Subscribe
Accepted Solution

reallocate on a metrocluster - split the mirror?!

Hi there

First of all, read the following threads:

http://communities.netapp.com/thread/7456

http://communities.netapp.com/message/20969#20969

Both threads we're informative, but the solution to decrease the latency does not work on our systems.

We've got two FAS6030 in a metrocluster configuration on 7.3.2P2. As the metro is using local_syncmirror and we are using snapshots (nearly everywhere :-)) i get the following:


US-S001MUR> reallocate status /vol/esxprod2_nfs_sys 
Reallocation scans are on
/vol/esxprod2_nfs_sys:
        State: Idle
     Schedule: n/a
     Interval: 1 day
Optimization: 5 [hot-spots: 26]

US-S001MUR> reallocate start -f -p /vol/esxprod2_nfs_sys
Unable to start reallocation scan on '/vol/esxprod2_nfs_sys': Aggregate is mirrored
US-S001MUR> reallocate start -f /vol/esxprod2_nfs_sys  
Unable to start reallocation scan on '/vol/esxprod2_nfs_sys': target has snapshots

whats the easiest way to reallocate on our configuration?

Thank you in advance!

regards,

Adrian

Re: reallocate on a metrocluster - split the mirror?!

I have run into the same issue with a test system. Like you the only option I could think of is destroying the mirror, reallocate and then build up the mirror from scratch again.

Re: reallocate on a metrocluster - split the mirror?!

Oh, that sounds really painfull.

does anyone know the reason for this limitation?

What do you recommend to do? Is it possible to split an aggr, reallocate about 150 volumes and resync the aggregate within 4 days?

As I have no metrocluster to test with, i would be thankful for every advice :-)

kind regards,

adrian

Re: reallocate on a metrocluster - split the mirror?!

My guess is it has something to do with the syncmirror snapshot on aggregate level. Reallocating with the -p option on volume level will still cause a large aggregate snapshot.

No idea how long reallocating will take. Perhaps netapp support can give you an estimate.

Re: reallocate on a metrocluster - split the mirror?!

Any further details on this from NetApp by any chance? Especially when running VMware environments with dedupe, I find reallocate to be critical....so losing it when running MetroCluster (as I have one customer doing) is a serious issue IMHO.

Re: reallocate on a metrocluster - split the mirror?!

Andrew Miller wrote:

Any further details on this from NetApp by any chance?

Nothing, it is still on my very long to do list

Re: reallocate on a metrocluster - split the mirror?!

He he....I understand what long to do lists are like. I'd be curious to hear whatever you find out...

Re: reallocate on a metrocluster - split the mirror?!

our key account manager advised me to open a case because of that.

The support told me, that there's a request for enhancment, but - as usual - with no eta.

So I have to cross my fingers that I don't get caught by murphy's law while the plexes are splitted.

Re: reallocate on a metrocluster - split the mirror?!

Hi,

Two ideas:

(1) The command reallocate start -f /vol/esxprod2_nfs_sys will indeed give an error if you have snapshots (because of the high rate of changed blocks that might result in filling up your vol) but you can run it per LUN/file like reallocate start -f /vol/esxprod2_nfs_sys/data.vmdk.  Maybe give it a try on some VMs that you think are most impacted.

(2) If you have a metrocluster you can enable reads to hit both plexes as by default its set to local plex only.  If you have a long distance between sites, or know you'll saturate your ISLs in a fabric MC then no solution but for many situations it allows you to double your read IOPs at no cost.  The behavour is configured by a system-wide option, can be done online, and takes effect immediately:

filer> options raid.mirror_read_plex_pref alternate

From the manpage:


raid.mirror_read_plex_pref
Specifies the plex preference when reading from a mirrored traditional volume or aggregate on a metrocluster-configured system. There are three possible values -- `local' indicates that all reads are handled by the local plex (plex consisting of disks from Pool0), `remote' indicates that all reads are handled by the remote plex (plex consisting of disks from Pool1), and `alternate' indicates that the handling of read requests is shared between the two plexes. This option is ignored if the system is not in a metrocluster configuration, i.e., cluster_remote is not licensed. The option setting applies to all traditional volumes and aggregates on the filer.

Cheers,
Chris

Re: reallocate on a metrocluster - split the mirror?!

Thanks for the info -- very helpful. Is it safe to assume that "reallocate measure" works the same in MC as otherwise? (so could diagnose which LUNs would be helped by reallocate)