Dataset Migration

[kofchur]

Due to performance problems, I'm in the process of migrating on to a new install of DFM 4.0.2.  I need to recreated this instance exactly the same as the old.  So far I have been able to recreate all the groups and their members, resource pools, schedules, and policies, but when I get to the datasets there seems to be a big problem.  I recreated the same dataset config on the new instanace (with the same members, resource pools and even the volume within the resource pool) and suspended it.  Then, I suspended the dataset on the old instance and resumed it on the new.  Well, I was hoping that the new instance would pickup where the old dataset left off, but instead it reinitialized it and laid down a new baseline transfer.  This is not good.  It seems that the datasets are tied to the dfm server on which they were created.  How can I get past this?  I can't export the relationships on the old then import them because when I do this in production I will have thousands of relationships to deal with.  I need to be able to have the newly created datasets "take over" the control of the dataprotection from the old datasets seemlessly.  I have not been able to find any info on how to do this.  Thanks for the help.

--todd

[adaikkap · Accepted solution]

Hi Todd,

     You are true, a dataset is tied to a DFM server. The way to move a datatset from one dfm server to another is by relinquish the relationship in server 1 and import those relationship on server 2.

The BPG details on moving a relationship out of data, which is same when moving from one server to another.

https://kb.netapp.com/support/index?page=content&id=1013426

5.12 GRACEFULLY RETIRING RELATIONSHIPS FROM ONCOMMAND PROTECTION TOOL

In order to retire relationships (or to move relationships out of a dataset), administrators delete their datasets along with the dpReaperCleanupModer option set to Never. This is not a healthy

option. You need to follow these steps to retire a relationship gracefully from a dataset:

Relinquish the relationship – This will enable PM to mark the relationship as external

Use the dfpm reliniquish command

Remove the Primary Volume/Qtree from the dataset – Do not remove the secondary volume first as the conformance will trigger a new rebaseline. Remove the Secondary Volume from the dataset. Delete the dataset (if required)

Note: It is true that you can delete the dataset with dpreapercleanupmode set to “never” to avoid the deletion of relationships, but it needs to stay that way forever, if re-activated PM will try to

reap the relationships.

CONSEQUENCES OF DELETING A DATASET

One really needs to consider the possible consequences of deleting a dataset, as the operation cannot be undone. Deleting a dataset not only affects the relationships (through

dpReaperCleanUp mode), but you will end up losing the backup versions. The backup versions are index of snapshots of members in a dataset. These backup versions are responsible for

restore operations and snapshot expiration based on retention time and retention count.If the backup versions are lost (due to the deletion of a dataset) then you will end up having

orphaned snapshots that cannot be restored through OnCommand Protection tool and will never be expired occupying huge amounts of space over time

So follow 5.12, dont delete the dataset on server 1. Add the controllers src and dst to server 2 and import the relationship  in server 2  into a newly created dataset.

Regards

adai

[kofchur]

Thanks, Adai.

It would be nice if we could put functionality into OC in the future to allow for toggling back and forth between identical datasets on different DFM instances.  Now I have to deal with the relinquishing and importing of thousands of relationships into new datasets.  This would not be a big deal in a small environment, but for large enterprises this is tedious.

--todd

[adaikkap]

I agree todd, BUT  the dataset info is tied to a DFM server. And managing the same members from two dfm server will cause dueling effect.If you are bringing up a new instance, just importing the members from the external relationship into the dataset would be fine provided you scrap the older instance. But if you want both to exisit then you will have to do one of the two things.

1.Destory the dataset in older instance. or

2.Reliquish the relatinship and leave the dataset.

The 1st would reap the relationship within 2hours ,

where as the 2nd approach is better because it will mark the relationships as external and take care of deleting all the old snapshot as per the retention so that you dont lose any space on the filer.

Regards

adai

[kofchur]

Adai,  Thanks for the input.

I followed these steps, but after I relinquished and removed the members of the relationships out of the old dataset and imported them into the new dataset, I can't see the old backup versions from the old dataset.  I did not delete it:

# dfpm dataset snapshot list toddset
Id   Name            Unique Id    Volume          Timestamp            Versioned  Dependencies    % of Total Blocks
---- --------------- ------------ --------------- -------------------- ---------- --------------- ----------
26731407 ASALPOFAS005(1573753520)_toddbackup_toddset_ASALPOFAS002_toddtest01-src.0 1327091481   ASALPOFAS002:/toddtest01 20 Jan 2012 15:31:21 No         SnapMirro
r      0%  (0%)
26731408 ASALPOFAS005(1573753520)_toddbackup_toddset_ASALPOFAS002_toddtest02-src.0 1327092487   ASALPOFAS002:/toddtest02 20 Jan 2012 15:48:07 No         SnapMirro
r      0%  (0%)
26730469 ASALPOFAS005(1573753520)_toddbackup_toddset_ASALPOFAS002_toddtest01-dst.2 1327091390   ASALPOFAS005:/toddbackup 20 Jan 2012 15:29:50 Yes        Busy - Sn
apMirror 0%  (0%)
26730530 ASALPOFAS005(1573753520)_toddbackup_toddset_ASALPOFAS002_toddtest02-dst.2 1327092393   ASALPOFAS005:/toddbackup 20 Jan 2012 15:46:33 Yes        Busy - Sn
apMirror 0%  (0%)

todd.kofchur@ASALSITCWPSTO01 ~
# dfpm dataset list -R toddset
Id         Name                        Policy                      Provisioning Policy Relationship Id State        Status  Hours Source                       Des
tination
---------- --------------------------- --------------------------- ------------------- --------------- ------------ ------- ----- ---------------------------- ----------------------------
5717 toddset                     testpolicy                                                 5735 snapmirrored idle    0.9   ASALPOFAS002:/toddtest01/-   ASA LPOFAS005:/toddbackup/toddset_ASALPOFAS002_toddtest01
5717 toddset                     testpolicy                                                 5737 snapmirrored idle    0.7   ASALPOFAS002:/toddtest02/-   ASA
LPOFAS005:/toddbackup/toddset_ASALPOFAS002_toddtest02

todd.kofchur@ASALSITCWPSTO01 ~
# dfpm dataset relinquish ASALPOFAS005:/toddbackup/toddset_ASALPOFAS002_toddtest01
Relinquished relationship (5735) with destination toddset_ASALPOFAS002_toddtest01 (5734).

todd.kofchur@ASALSITCWPSTO01 ~
# dfpm dataset relinquish ASALPOFAS005:/toddbackup/toddset_ASALPOFAS002_toddtest02
Relinquished relationship (5737) with destination toddset_ASALPOFAS002_toddtest02 (5736).

<removed members of the relationships via GUI and imported them into new dataset>

todd.kofchur@ASALSITCWPSTO01 ~
# dfpm dataset snapshot list toddset
No Snapshot copies to list for the object toddset.

todd.kofchur@ASALSITCWPSTO01 ~
#

[adaikkap]

Hi Todd,

     For old backup versions you should go to the old dataset. As a backup version is meta data attached to the dataset and not to the volumes of the dataset.The old dataset will take care of retiring or expiring the snapshots based on the retention settings.The "CONSEQUENCES OF DELETING A DATASET" explains this and that's why we recommend not to delete the old dataset.

You will have to retain the old dataset for all restores from previous backup version or until they expire. Or as always you can use the snapshot which still exist on the volume to restore them.

Regards

adai

[kofchur]

Adai,

Again thanks for the answers.

I understand what you are saying but the problem that I'm having is that I can't access the snapshot data on the old dataset after I relinquish and move the members out.  I have tested this twice and each  time it appears that I can't access the old snapshots via DPM.  In the following test I relinquished and removed 2 volumes and their destination volume from their dataset.  I could see the old snapshots after I relinqished them, but after I removed them I can't see the old snapshots with the "dfpm dataset snapshot list" command.  I guess I'm missing as to how, without getting on the filer itself, I can acceess the old snapshots via the NMC:

# dfpm dataset list -R toddset
Id         Name                        Policy                      Provisioning Policy Relationship Id State        Status  Hours Source                       Des
tination
---------- --------------------------- --------------------------- ------------------- --------------- ------------ ------- ----- ---------------------------- ---
-------------------------
      5717 toddset                     testpolicy                                                 5748 snapmirrored idle    0.2   ASALPOFAS002:/toddtest03/-   ASA
LPOFAS005:/toddbackup/toddset_ASALPOFAS002_toddtest03
      5717 toddset                     testpolicy                                                 5750 snapmirrored idle    0.2   ASALPOFAS002:/toddtest04/-   ASA
LPOFAS005:/toddbackup/toddset_ASALPOFAS002_toddtest04

todd.kofchur@ASALSITCWPSTO01 ~
# dfpm dataset relinquish ASALPOFAS005:/toddbackup/toddset_ASALPOFAS002_toddtest03
Relinquished relationship (5748) with destination toddset_ASALPOFAS002_toddtest03 (5747).

todd.kofchur@ASALSITCWPSTO01 ~
# dfpm dataset relinquish ASALPOFAS005:/toddbackup/toddset_ASALPOFAS002_toddtest04
Relinquished relationship (5750) with destination toddset_ASALPOFAS002_toddtest04 (5749).

todd.kofchur@ASALSITCWPSTO01 ~
# dfpm dataset snapshot list toddset
Id   Name            Unique Id    Volume          Timestamp            Versioned  Dependencies    % of Total Blocks
---- --------------- ------------ --------------- -------------------- ---------- --------------- ----------
27059965 2012-01-23 16:30:05 hourly_ASALPOFAS002_toddtest03.- 1327336205   ASALPOFAS002:/toddtest03 23 Jan 2012 11:30:05 Yes        None            0%  (0%)
27062511 dfpm_base(toddset.5717)conn1.0 1327336569   ASALPOFAS002:/toddtest03 23 Jan 2012 11:36:09 No         SnapMirror      0%  (0%)
27062498 2012-01-23 17:00:05 hourly_ASALPOFAS002_toddtest03.- 1327338005   ASALPOFAS002:/toddtest03 23 Jan 2012 12:00:05 Yes        None            0%  (0%)
27064968 2012-01-23 17:30:05 hourly_ASALPOFAS002_toddtest03.- 1327339806   ASALPOFAS002:/toddtest03 23 Jan 2012 12:30:06 Yes        None            0%  (0%)
27059966 2012-01-23 16:30:05 hourly_ASALPOFAS002_toddtest04.- 1327336209   ASALPOFAS002:/toddtest04 23 Jan 2012 11:30:09 Yes        None            0%  (0%)
27062514 dfpm_base(toddset.5717)conn1.0 1327336567   ASALPOFAS002:/toddtest04 23 Jan 2012 11:36:07 No         SnapMirror      0%  (0%)
27062499 2012-01-23 17:00:05 hourly_ASALPOFAS002_toddtest04.- 1327338009   ASALPOFAS002:/toddtest04 23 Jan 2012 12:00:09 Yes        None            0%  (0%)
27064969 2012-01-23 17:30:05 hourly_ASALPOFAS002_toddtest04.- 1327339814   ASALPOFAS002:/toddtest04 23 Jan 2012 12:30:14 Yes        None            0%  (0%)
26735581 2012-01-20 21:38:13 daily_ASALPOFAS005_toddbackup.-.toddset_ASALPOFAS002_toddtest01.toddset_ASALPOFAS002_toddtest02 1327095399   ASALPOFAS005:/toddbackup
20 Jan 2012 16:36:39 No         None            0%  (0%)
26994320 2012-01-23 03:01:30 weekly_ASALPOFAS005_toddbackup.-.toddset_ASALPOFAS002_toddtest01.toddset_ASALPOFAS002_toddtest02 1327287597   ASALPOFAS005:/toddbacku
p 22 Jan 2012 21:59:57 No         None            0%  (0%)
27065942 ASALPOFAS005(1573753520)_toddbackup_toddset_ASALPOFAS002_toddtest03-dst.6 1327340078   ASALPOFAS005:/toddbackup 23 Jan 2012 12:34:38 No         Busy - Sn
apMirror 0%  (0%)
27065941 ASALPOFAS005(1573753520)_toddbackup_toddset_ASALPOFAS002_toddtest04-dst.6 1327340080   ASALPOFAS005:/toddbackup 23 Jan 2012 12:34:40 No         Busy - Sn
apMirror 0%  (0%)
27065940 ASALPOFAS005(1573753520)_toddbackup_toddset_ASALPOFAS002_toddtest01-dst.10 1327340081   ASALPOFAS005:/toddbackup 23 Jan 2012 12:34:41 No         Busy - S
napMirror 0%  (0%)
27065939 ASALPOFAS005(1573753520)_toddbackup_toddset_ASALPOFAS002_toddtest02-dst.10 1327340082   ASALPOFAS005:/toddbackup 23 Jan 2012 12:34:42 No         Busy - S
napMirror 0%  (0%)
27065876 2012-01-23 17:37:19 hourly_ASALPOFAS005_toddbackup.-.toddset_ASALPOFAS002_toddtest01.toddset_ASALPOFAS002_toddtest02... 1327340136   ASALPOFAS005:/toddba
ckup 23 Jan 2012 12:35:36 Yes        None            0%  (0%)
27065937 2012-01-23 17:37:21 hourly_ASALPOFAS005_toddbackup.-.toddset_ASALPOFAS002_toddtest01.toddset_ASALPOFAS002_toddtest02... 1327340138   ASALPOFAS005:/toddba
ckup 23 Jan 2012 12:35:38 No         None            0%  (0%)

<AT THIS TIME I REMOVED SOURCE AND DESTINATION VOLUMES/QTREES VIA NMC>

todd.kofchur@ASALSITCWPSTO01 ~
# dfpm dataset snapshot list toddset
No Snapshot copies to list for the object toddset.

[adaikkap]

Hi Todd,

           In order to restore from the old dataset use the NMC restore wizard. The snapshot can be listed in the new dataset or use the dfpm backup list on the old dataset to see the old backup

version.

Regards

adai