OnCommand Protection Manager SV solution for more than 250 snapshots

[jhubert]

Is there a recommended process for handling more than 250+ snapshots in Protection Manager?  Specifically transitioning a Protection Manager Dataset to a new volume for SnapManager for Exchange/SQL SnapVaults?  I understand this would be a manual process.

In this environment the requirement is to maintain a year worth of snapshots - although I have recommended 6 months of Daily, 6 months of Weekly, etc..(Max of 250). 

Therefore I need to develop a process for gracefully transitioning to a new volume.  In my testing I've found that if I remove the destination SV volume, and then 'Conform' the dataset, it will create a new volume from the resource pool and continue the SV relationship.  I then have to issue a 'dfbm primary dir relinquish <ID>' to stop OnCommand/PM errors about unable to remove SnapVault relationship. 

At this point, I've noticed that I still retain the ability to restore from the previous volume through the restore button in OnCommand/PM.

Do I need to also remove the relationship from PM via 'dfpm dataset relinquish <ID>'?  Would that prevent the previous snapshots from being seen during a restore operation? 

Workflow:

SME/SMSQL => Created OnCommand/PM dataset

PRI_FILER1 => SEC_FILER1

SME jobs create local snapshots (7 days)

SME causes OnCommand/PM SnapVault update.

--

Once backup volume reaches 250 snapshots, the relationship needs to be moved to a new volume while retaining the ability to restore from the old backup volume.

(?) Remove Destination Volume from Dataset

(?) Release Volume with 'dfbm primary dir relinquish <ID>'

(?) Conform Dataset to SnapVault to new Volume

(?) Continue process until total of 3 volumes, 1 active SV.  Remove oldest Volume once snapshots in 2 recent volumes totals 356 days.

--

ADDITIONAL QUESTIONS

(?) Break SV relationship between PRI and old SV volume (?) - Would this affect restore in PM?

Thanks,

Jeff

[adaikkap · Accepted solution]

Hi Jeff,

          Here is the details steps on how to achieve the same.

My dataset details:

Dataset Details:

++++++++++++

C:\>dfpm dataset list -x 462

Id:                              462

Name:                            moreThan255

Protection Policy:               Back up

Application Policy:

Description:

Owner:

Contact:

Volume Qtree Name Prefix:

Snapshot Name Format:            %T

Primary Volume Name Format:

Secondary Volume Name Format:

Secondary Qtree Name Format:

DR Capable:                      No

Requires Non Disruptive Restore: No

Node details:

   Node Name:           Primary data

   Resource Pools:

   Provisioning Policy:

   Time Zone:

   DR Capable:          No

   vFiler:

   Node Name:           Backup

   Resource Pools:      mpovsim14Rp

   Provisioning Policy: dedupe

   Time Zone:

   DR Capable:          No

   vFiler:

Primary and Secondary Volume Details:

+++++++++++++++++++++++++++++++++++

C:\>dfpm dataset list -m 462

Id         Node Name            Dataset Id Dataset Name         Member Type                                        Name

---------- -------------------- ---------- -------------------- -------------------------------------------------- --------------------------------------

       446 Primary data                462 moreThan255          volume                                             mpo-vsim13:/noVolLang

       464 Backup                      462 moreThan255          volume                                             mpo-vsim14:/noVolLang

Relationship Details:

++++++++++++++++++

C:\>dfpm dataset list -R 462

Id         Name                        Protection Policy           Provisioning Policy Relationship Id State        Status  Hours Source                       Destination                

---------- --------------------------- --------------------------- ------------------- --------------- ------------ ------- ----- ---------------------------- ----------------------------

       462 moreThan255                 Back up                     468 snapvaulted  idle    0.1   mpo-vsim13:/noVolLang/two    mpo-vsim14:/noVolLang/two

       462 moreThan255                 Back up                     470 snapvaulted  idle    0.1   mpo-vsim13:/noVolLang/one    mpo-vsim14:/noVolLang/one

       462 moreThan255                 Back up                     472 snapvaulted  idle    0.1   mpo-vsim13:/noVolLang/-          mpo-sim14:/noVolLang/moreThan255_mposim13_noVolLang

Now the steps to get new secondary volume created:

+++++++++++++++++++++++++++++++++++++++++++++++

1. Relinquish the relationships using dfpm dataset relinquish cli so that PM stops scheduling the relationships anymore
2. Remove the secondary volume usings dfpm dataset remove cli, this way the old secondary volume where we reached 250 snapshot is removed and a new secondary volume is created with re-baseline from primary.
3. This way you still have the relationship between your primary and old secondary( though this is not a requirement for PM to restore, also PM will no more update this relationship).

STEP 1:

          C:\>dfpm dataset relinquish mpo-vsim14:/noVolLang/moreThan255_mpo-vsim13_noVolLang

               Relinquished relationship (472) with destination moreThan255_mpo-vsim13_noVolLang (471).

         C:\>dfpm dataset relinquish mpo-vsim14:/noVolLang/one

                Relinquished relationship (470) with destination one (469).

          C:\>dfpm dataset relinquish mpo-vsim14:/noVolLang/two

              Relinquished relationship (468) with destination two (467).

As you can see that the dataset no more shows up this relationship.

C:\>dfpm dataset list -R 462

Id         Name                        Protection Policy           Provisioning Policy Relationship Id State        Status  Hours Source                       Destination

---------- --------------------------- --------------------------- ------------------- --------------- ------------ ------- ----- ---------------------------- ----------------------------

C:\>

STEP 2:

C:\>dfpm dataset remove -N Backup 462 mpo-vsim14:/noVolLang

Dataset dry run results

----------------------------------

Do: Provision flexible volume (backup secondary) of size 26.4 MB

Effect: Provision a new flexible volume of 26.4 MB from aggregate 'mpo-vsim14:aggr1'(321).

Do: Enable deduplication on flexible volume.

Effect: Enable deduplication on flexible volume 'VolToBeProvision:moreThan255' (29)

Do: Create backup relationship(s) for dataset 'moreThan255' (462) on connection 1.

Effect: Create backup relationship(s) between 'mpo-vsim13:/noVolLang/two' and new volume to be provisioned from resource pool(s) 'mpovsim14Rp' (396).

Create backup relationship(s) between 'mpo-vsim13:/noVolLang/one' and new volume to be provisioned from resource pool(s) 'mpovsim14Rp' (396).

Create backup relationship(s) between 'mpo-vsim13:/noVolLang/-' and new volume to be provisioned from resource pool(s) 'mpovsim14Rp' (396).

Removed volume mpo-vsim14:/noVolLang (464) from dataset moreThan255 (462).

C:\>

If you wish to check what this cli will do before actually running it,you can do a dry run with the switch -D

C:\>dfpm dataset remove -D -N Backup 462 mpo-vsim14:/noVolLang

Dataset dry run results

----------------------------------

Do: Provision flexible volume (backup secondary) of size 26.4 MB

Effect: Provision a new flexible volume of 26.4 MB from aggregate 'mpo-vsim14:aggr1'(321).

Do: Enable deduplication on flexible volume.

Effect: Enable deduplication on flexible volume 'VolToBeProvision:moreThan255' (21)

Do: Create backup relationship(s) for dataset 'moreThan255' (462) on connection 1.

Effect: Create backup relationship(s) between 'mpo-vsim13:/noVolLang/two' and new volume to be provisioned from resource pool(s) 'mpovsim14Rp' (396).

Create backup relationship(s) between 'mpo-vsim13:/noVolLang/one' and new volume to be provisioned from resource pool(s) 'mpovsim14Rp' (396).

Create backup relationship(s) between 'mpo-vsim13:/noVolLang/-' and new volume to be provisioned from resource pool(s) 'mpovsim14Rp' (396).

STEP3:

Now you can see only the relationship  of the new secondary volume.

C:\>dfpm dataset list -R 462

  Id         Name                        Protection Policy           Provisioning Policy Relationship Id State        Status  Hours Source                       Destination                

---------- --------------------------- --------------------------- ------------------- --------------- ------------ ------- ----- ---------------------------- ----------------------------

       462 moreThan255                 Back up           477 snapvaulted  idle    0.0   mpo-vsim13:/noVolLang/one    mpo-vsim14:/noVolLang_1/one

       462 moreThan255                 Back up           480 snapvaulted  idle    0.0   mpo-vsim13:/noVolLang/two    mpo-vsim14:/noVolLang_1/two

       462 moreThan255                 Back up           481 snapvaulted  idle    0.0   mpo-vsim13:/noVolLang/-      mpo-vsim14:/noVolLang_1/moreThan255_mpo-vsim13_noVolLang

You can see the relationship of the primary with the old secondary and new secondary using dfpm relationship list cli.

C:\>dfpm relationship list

Relationship Id Relationship Type    Dataset Id Dataset Name         Source                                   Destination                              Deleted Deleted By

--------------- -------------------- ---------- -------------------- ---------------------------------------- ---------------------------------------- ------- ----------

             468 snapvault                     0                      mpo-vsim13:/noVolLang/two                mpo-vsim14:/noVolLang/two              No

            470 snapvault                     0                      mpo-vsim13:/noVolLang/one                mpo-vsim14:/noVolLang/one              No

            472 snapvault                     0                      mpo-vsim13:/noVolLang/-                  mpo-vsim14:/noVolLang/moreThan255_mpo-vsim13_noVolLang No

            477 snapvault                   462 moreThan255          mpo-vsim13:/noVolLang/one                mpo-vsim14:/noVolLang_1/one              No

            480 snapvault                   462 moreThan255          mpo-vsim13:/noVolLang/two                mpo-vsim14:/noVolLang_1/two              No

            481 snapvault                   462 moreThan255          mpo-vsim13:/noVolLang/-                  mpo-vsim14:/noVolLang_1/moreThan255_mpo-vsim13_noVolLang No

You can also see all the Backup Version of both the new and old secondary volume and use the Restore Wizard as well.

C:\>dfpm backup list 462

Backup Id Backup Version        Retention Type Retention Duration (in seconds)   Node Name            Description                         Properties(Name=Value)

--------- --------------------- -------------- --------------------------------- -------------------- ----------------------------------- -------------------------

      278 10 May 2013 01:30:55  monthly        8467200                           Backup

      275 10 May 2013 01:29:15  monthly                                          Primary data

      276 10 May 2013 01:28:14  weekly         4838400                           Backup

      274 10 May 2013 01:28:10  weekly         1209600                           Primary data

      272 10 May 2013 01:11:09  daily          1209600                           Backup

      271 10 May 2013 01:11:05  daily          604800                            Primary data

      270 10 May 2013 01:05:08  daily          1209600                           Backup

      269 10 May 2013 01:05:05  daily          604800                            Primary data

      268 10 May 2013 01:01:19  unlimited                                        Backup

BTW we use API restore in case of Application dataset and its better to leave the relationship with the old secondary volume. In most cases we use NDMP for restore.

Hope this helps.

Regards

adai

[jhubert]

Thanks, this is very helpful.

What is the difference between dfbm primary dir relinquish and dfpm dataset remove/relinquish?

dfbm primary dir relinquish seems to also remove the relationship from the dataset. 

[adaikkap]

Hi Jeff,

     dfbm primary dir relinquish removes the entry from the database, but the next snapvault monitoring cycle will rediscover it again. Whereas dfpm dataset relinquish will make the dataset to relinquish its control over it like

• scheduling the same
• doing conformance check on the same

Using dfbm will lead to dataset showing redundant relationship error.

dfpm dataset remove, removes the secondary volume from the dataset. Then conformance kicks in and finds that there is a primary and no secondary but as per the protection policy this needs one. So it goes head and provisions and new secondary volume and does the re-baseline from primary.

Hope this helps.

Regards

adai

[jhubert]

Is there a NetApp Management Console GUI process for when the user removes a destination volume from the dataset? 

What I've found is that removing the destination volume in the NMC PM GUI (Backup - SnapVault), causes the conform process to create a new volume. 

However the customer still sees an error logged about "SnapVault Relationship Delete Failed" - some because of too many snapshots, and some because of PATH error.  To stop those from reoccuring, I've had to use the dfbm primary dir relinquish command to remove the relationship. The relationship disappears from the dfpm command.

In 2 years, the customer logged 300,000 error events for "SnapVault Relationship Delete Failed"!

Thanks,

Jeff

[adaikkap]

Hi Jeff,

     The reason you get the event is that you didnt relinquish the relationship from your dataset. The dfpm dataset remove is same as removing destination volume from NMC.

Pls use dfpm dataset relinquish first then followed by NMC removal of secondary volume or dfpm dataset remove both are same.

BTW use version 5.2, there is feature that purges all events older than 180days or value set in eventPurgeInterval.

Regards

adai

[KENBUNCE0]

this is a very helpful