How to migrate 32bit volume with qtree and non qtree data to 64 bit volume

[JLundfelt]

Hi

I have a 11TB CIFS based NTFS volume with 4 qtree's on 7.3.7 controller, that I need to migrate to another controller, the problem I am having is that

if I use QSM-

Invoke-NaSnapmirrorInitialize -Source 172.16.1.1:/vol/Groups/- -Destination irv-gdc-san1a:/vol/Groups/qtree

I only get the base contents of the volume, and not the other qtree's that are co-mingled with folders in the base of the cifs share local path (/vol/Groups)

if I use VSM-

The volume would be replicated with all data / qtree's. but would remain 32bit.

Part of the issue I have is that this is over 10TB / 10 million files. I already have the QSM relationship for the base contents of the volume. If I setup additional QSM's, they would be adjacent to the base contents, and a single cifs share could not access. As part of the cutover plan, I am going to be aliasing the IP address since we have both Windows and Unix clients accessing this data from god knows where. Anyone have any suggestions? I am trying to make this a single cutover event, where the outage is less than an hour, and from previous expireance, if I move the contents from the base QSM  (irv-gdc-san1a:/vol/Groups/qtree --> irv-gdc-san1a:/vol/Groups) it could take hours, and I wouldn't be able to maintain permissions unless I used robocopy with the /MIR and /SEC switches.

Any suggestions? TIA!

-jon

[aborzenkov]

Can you update to 8.1 or newer?

[JLundfelt]

No, unfortunately the source is a single controller so I can't do a NDU, or take an outage...

[JLundfelt]

I should clarify snapmirror target is HA pair of 3140's running 8.1.7p4, and as an end result I am trying to get everything to 64 bit volumes.

Thanks,
Jon

[dirk_ecker]

Hi Jon,

Is the destination aggregate on the FAS3140 a 64 bit aggregate? If so, you can simply perform a VSM migration from the 32 bit source aggr, the volume will be expanded to 64 bit automatically after breaking the snapmirror relationship.

If the destination aggregate is 32 bit and the FAS3140 controllers are running 8.1.4P4 or later you can perform an in-place aggregate extension without adding disks. Check out http://www.netapp.com/us/media/tr-3978.pdf page 7 for details.

I hope this helps!

Cheers,

d.

[JLundfelt]

Destination controller is 3140 running 8.1.4P8, with 64 bit aggregates. I swear I have seen VSM destinations stay 32 bit like their source, but I guess I dont have a VSM to test that theory with. Either way it sounds like the approach  I am going to have to take as at least a first step.  At least I know not to use QSM, since it can't keep the folder heirearchy in my scenario.

Thanks!

-Jon

[aborzenkov]

---

I swear I have seen VSM destinations stay 32 bit like their source

---

Conversion starts after snapmirror break; did you do it?

[JLundfelt]

It's an 11TB volume, and I am cleaning up the first 3TB base volume contents that was QSM'd right now. I trust everyone, and don't feel the need to test this theory, and again I have to use VSM regardless due to the qtrees mixed with folders heirarchy.. I will update the thread once its complete in a couple days, I need to take an outage on the target this morning to add some quad GbE NICs since we wont be able to use the 10gb interfaces for some time.

Thanks everyone. Update to follow

-Jon

[dirk_ecker · Accepted solution]

The VSM destination vol won't change until you break the snapmirror relationship.

For testing purposes you can do the following:

- create a new vol on the source 7.3.7 controller

- set up a VSM relationship to the FAS3140

- running "vol status" should show 32-bit now

- break the relationship

- wait a few minutes, then run "vol status" once again

[Darkstar]

VSM destinations will only be converted to 64bit after you break them.

[aborzenkov]

You are right of course. I do not know whay I was sure we need 8.1 source to perform 32 to 64 bit VSM; it is only reverse sync that requires it. Thank you!

[aborzenkov]

If you could not take an outage, you would not single controller. If you update source to 8.1 you can use VSM from 32 to 64 bit.

[JLundfelt]

What about just migrating the volume a second time to another aggregate with VSM once on target controller? So 32bit volume on single 7.3.7 controller migrated with VSM to 32bit volume on target controller, then subsequent VSM / 64 bit conversion?

[aborzenkov]

Yes, that's possible. Do not forget, that in-place conversion without increasing aggregate size beyond 16T is officially supported starting with 8.2.1 if my memory serves me right.

[richard_payne]

"if I use QSM-

Invoke-NaSnapmirrorInitialize -Source 172.16.1.1:/vol/Groups/- -Destination irv-gdc-san1a:/vol/Groups/qtree

I only get the base contents of the volume, and not the other qtree's that are co-mingled with folders in the base of the cifs share local path (/vol/Groups)"

So the '-' will only get the non-qtree data, yes, but you could also setup QSM for the other qtrees (4 I think you said). So you'd end up with 5 qtree snapmirrors. This would also give you the advantage of converting the non-qtree data into a qtree.

I see you're going down the VSM route (which also makes sense)....but I don't see the problem with using QSM for this, maybe I'm missing something?

--rdp

[JLundfelt]

The problem is with the way the cifs share points at the root of the volume, and there are qtree's with data as well. If I moved them all w/ QSM, the folder heirarchy wouldn't work-

> get-naqtree *groups*

Volume Qtree Status Security
------ ----- ------ --------
Groups normal ntfs
Groups Programming normal ntfs
Groups Information Services normal ntfs
Groups Product Design & Dev normal ntfs

> Get-NaCifsShare Groups

MountPoint        ShareName   Description
----------            ---------            -----------
/vol/Groups        Groups           Document Storage

So the local share mask would need to point to the base of the volume, where what used to be the folder in the root of the volume would be in a qtree.

[richard_payne]

I see, and the users are expecting the 4 qtrees to be under the top level share....makes sense.

--rdp

[JLundfelt]

So one of the only issues I am having with this ~10TB VSM is that even though the baseline completed in about 72 hours, the differentials, which are scheduled to run every 10 minutes-

Source                      Destination               Minutes               Hours DaysOfWeek DaysOfMonth
------                         -----------                    -------                 ------- ----- ---------- -----------
172.16.1.1:Groups   irv-gdc-san1a:Groups 0,10,20,30,40,50   *         *         *           *

End up taking anywhere between 5-12 hours to complete each transfer, and during that time the transferprogress shows no data being transferred-

BaseSnapshot : irv-gdc-san1a(0151769424)_Groups.49
Contents : Replica
CurrentTransferError :
CurrentTransferType : scheduled
DestinationLocation : irv-gdc-san1a:Groups
InodesReplicated :
LagTime : 30597
LagTimeTS : 08:29:57
LastTransferDuration : 18789
LastTransferDurationTS : 05:13:09
LastTransferFrom : 172.16.1.1:Groups
LastTransferSize : 325738496
LastTransferType : scheduled
MirrorTimestamp : 1437984008
MirrorTimestampDT : 7/27/2015 1:00:08 AM
ReplicationOps :
SourceLocation : 172.16.1.1:Groups
State : snapmirrored
Status : transferring
TransferProgress : 0
InodesReplicatedSpecified : False
ReplicationOpsSpecified : False
Source : 172.16.1.1:Groups
Destination : irv-gdc-san1a:Groups

I though that VSM was faster since it didnt operate at the file/inode level, but at the block level. Is this simply an issue of the size of the source volume, and the snapmirror process taking hours to scan for changes? The current source volume is protected with DFPM, and has hourly snapshots, and nightlight snapvaults to another off-site NetApp. Any thoughts on this, or should I just open a support case since this is going to severly limit the control over the timing of the cutover.

Thanks,

Jon

[richard_payne]

I would open a case, yes. How busy are the two controllers?

I would expect VSM to be much faster than QSM in this case....

--rdp

[scottgelb]

What is the change between snapshot updates using the "snap delta" command?  Also the output of "snapvault status -l" to see the detailed listing.  Also could be something in options replication.throttle if set at the controllers or something in the Protection Manager job.  From the controller side it would be good to see the output of those 3 commands on both sides.

[JLundfelt]

Yeah, I just opened a case. I think the source controller is definately busy, although its not as simple as just the CPU utilzation. Trying to do a snap list, let alone a snap delta against the source volume in question for the past 30 minutes, and am still waiting, and cross fingers that it doesnt panic the controller. I can't enumerate the snaps with the shell, PowerShell Toolkit, or Filerview (source controller is FAS3140 running 7.3.7) and will hopefully be decomissioned as soon as I can get a defined cutover schedule.

Thanks,

Jon