Subscribe
Accepted Solution

How to protect a volume with ~1000 Qtrees with PM?

Hi folks,

I'm currently trying to set up a customer demo that involves PM-managed backups.

Caveat: The source volume contains 996 Qtrees. The Dataset never reaches conformance as PM fails to create more than 253 relationships as the source volumes runs out of snapshots.

Any idea how to protect volumes with >250 Qtrees?

Also all Qtrees from a single volume should be SV'ed to the same secondary volume, hence I have to set the following option accordingly:

pmMaxSvRelsPerSecondaryVol   1000

I know it's far beyond the default 50, but should I expect any negative impact?

regards, Niels

Re: How to protect a volume with ~1000 Qtrees with PM?

I'm successfully doing this with about 200 qtrees. I don't know why PM is trying to create so many snaps on the primary. What you may need to do is build the dataset gradually by adding a smaller number of qtrees at a time, waiting for conformance to complete before adding more.

Cheers,

Richard

Re: How to protect a volume with ~1000 Qtrees with PM?

Hi Niels,

     Yes you can do it. For each qtree on the primary snapvault creates a base snapshot, on first update all off them are coalesced into one base snapshot. Below is a example.

fas-sim-1> qtree status OneThousand

Volume   Tree     Style Oplocks  Status

-------- -------- ----- -------- ---------

OneThousand          unix  enabled  normal

OneThousand one      unix  enabled  normal

OneThousand three    unix  enabled  normal

OneThousand two      unix  enabled  normal

fas-sim-1>

After SnapVault Start/Create Relationship job

fas-sim-1> snap list OneThousand

Volume OneThousand

working...

  %/used       %/total  date          name

----------  ----------  ------------  --------

21% (21%)    0% ( 0%)  Apr 15 11:31  fas-sim-2(0099931872)_OneThousand_backup_one-src.0 (snapvault)

36% (23%)    0% ( 0%)  Apr 15 11:31  fas-sim-2(0099931872)_OneThousand_backup_OneThousand_fas-sim-1_OneThousand-src.0 (snapvault)

46% (23%)    0% ( 0%)  Apr 15 11:31  fas-sim-2(0099931872)_OneThousand_backup_two-src.0 (snapvault)

53% (21%)    0% ( 0%)  Apr 15 11:31  fas-sim-2(0099931872)_OneThousand_backup_three-src.0 (snapvault)

fas-sim-1>

After SnapVault Update/Protect Now

fas-sim-1> snap list OneThousand

Volume OneThousand

working...

  %/used       %/total  date          name

----------  ----------  ------------  --------

27% (27%)    0% ( 0%)  Apr 15 11:39  dfpm_base(OneThousand.436)conn1.0 (snapvault,acs)<<<<<<<<<<<<<<<<<<<<<<<<<SV Base Snapshot with dataset name & id

39% (21%)    0% ( 0%)  Apr 15 11:38  2012-04-16 12:40:54 daily_fas-sim-1_OneThousand.-.one.three.two<<<<<<<<<<<<<<<<Backup snapshot created by Protect Now.

fas-sim-1>

As the max snapshot per volume is 255, after creating 255 qtree snapvault relationships the dataset will become non-conformant with error saying no snapshot available.

Now run a Protect Now from Protection Manager, all this 255 will be coalesce into one. But still the dataset will show the conformance status as non-conformant.Click on the same and say conform now.

PM will now create relation for next 253 qtree ( as one is already used by dfpm_base and other by the backup snapshot of PM).Once this is done, again it will fail due to non availability of snapshot.

Run Protect now. Keep doing the same until all 1000 qtrees are snapvaulted.

The down side is that, max concurrent SV stream per controller is limited and various with the following.

ONTAP Version

FAS Model

NearStore License being enabled or not.


The regular scheduled updates of this volume, will consume all SV threads until its finished and can increase the back window and delay  snapshot creation on the secondary as alll 1000 needs to be snapvaulted before a SV snapshot can be created on the destination. This is the only downside I can think of.

This limit for 50 was done mainly for QSM as each qtree in a QSM needs a base snapshot and only remaining 205 would be available for long term retention as max snapshots per volume is only 255.

Also do remember the options you are changing is a global option and applies to all dataset creating SV relationship.

Regards

adai

Regards

adai

Re: How to protect a volume with ~1000 Qtrees with PM?

Hi Richard,

thanks for that suggestion.

Although adding qtrees one at a time might be helpful, it's not making it very convenient.

The reason about using PM for relationship management is *not* to care about individual resources.

The idea is to add whole volumes, or even the containing aggregate to the dataset and let PM do it's magic.

Otherwise I'd have to regularly check if new Qtrees have been created that are not getting protected and I'd have to add them manually to the dataset. That's not how I expect PM to work.

regards, Niels

Re: How to protect a volume with ~1000 Qtrees with PM?

Thanks Adai. That sounds as if it's at least doable, although I'd expect PM to handle that on it's own.

I'll go ahead testing and rate the answer accordingly once it's finished (which could take a while as I'm using a FAS270 which is capable of running just seven SV relationships at a time...)

regards, Niels

Re: How to protect a volume with ~1000 Qtrees with PM?

I agree with what you said. I am not sure if you can just add a volume into PM and have it watch for new qtrees though - having said that it's not something I've tested and it may work. Adai - can you comment on this?

Re: How to protect a volume with ~1000 Qtrees with PM?

If the Dataset contains a volume, PM will pick up any newly created qtree automatically during the next conformance run and create the destination qtree and the relationship automagically. That's the whole beauty about PM.

If you add a whole aggregate or even a controller, this happens to all underlaying volumes. No need to hassle with individual qtrees/volumes.

regards, Niels

Re: How to protect a volume with ~1000 Qtrees with PM?

Hi Neils,

     I have done it many time with other customer who had more than 255 like  300+ and not 1000.

Regards

adai

Re: How to protect a volume with ~1000 Qtrees with PM?

Hi

     In fact you can even add an entire filer, its called indirect referencing.Though at the end of the day the relationship are created at the qtree or volume level depending upon the replication technology. When an entire filer is added to the primary of  a dataset, PM knows what are all the volume and its containing qtrees in the filer. Once you commit your dataset PM kick off creating relationship for each of them as per the technology( VSM/QSM/SV). PM takes its data from the dfm db, which discovers for new volumes or qtrees once every 15minutes by default. When conformance run on the dataset once every 1 hour by default checks for primary members like its qtree and volumes( irrespective of what is the  direct member of the dataset, like volume/aggr/filer) and check its secondary to see if there is a corresponding relationship if not it kicks off create relationship jobs. This is the one of the sole job of conformance engine.

Regards

adai

Re: How to protect a volume with ~1000 Qtrees with PM?

Excellent - thank you for clearing that up. As a side note, do you have an ETA when DFM 5.0.1 will be released?