ONTAP Discussions

Verify on SnapVault secondary gets slower every month

mheimberg
3,483 Views

We are archiving snapshots of a 600+ GB MS SQL database by means of Protection Manager to a FAS2050 as SnapVault secondary system.

To save the primary systems resources we run the verify from the secondary system.

This is working well and satisfied everyone. But now we realize that each month the verification is taking plus 1 hour - starting from 8 hrs in the beginning to over 15 hrs at the moment - so in a couple of months it won't fit into the backup window anymore.

The DB is not changing or growing in an extraordinary way, at least far away from double the size, so what could be the reason for this?

THX for the inputs.

Mark

6 REPLIES 6

aborzenkov
3,483 Views

Fragmentation? What is the result of "reallocate measure" for volume on secondary system? You could run it periodically for some time to see trend.

It would also be interesting to run (at least once) "reallocate measure" for each individual LUN, as some of them could be more fragmented than others.

mheimberg
3,483 Views

I will check with the suggested reallocate measure - but it will propably take some time.

thx

Mark

mheimberg
3,483 Views

The results of the scan or quite obvious I think:

Reallocation scans are on
/vol/SnapMgr_SQLServer_CHDB008_backup/db/iscsi_db008_e.lun:
        State: Idle
     Schedule: n/a
     Interval: 1 day
Optimization: 11
  Measure Log: n/a

But when I try to start a reallocate, the system complains:

ntap> reallocate start -o -n /vol/SnapMgr_SQLServer_CHDB008_backup/db/iscsi_db008_e.lun
Tue Dec 21 16:37:54 CET [wafl.scan.start:info]: Starting file reallocating on volume SnapMgr_SQLServer_CHDB008_backup.
Tue Dec 21 16:37:54 CET [wafl.scan.reallocDisallowed:warning]: Reallocation disallowed on inode 54125779 in volume SnapMgr_SQLServer_CHDB008_backup: readonly or snapshot.

So we have a very unoptimized LUN, but are unable to do anything about it, because it is a SnapVault secondary

An option would be to quiesce the SnapVault, convert it into a SnapMirror, break the mirror, reallocate, re-establish the mirror, re-convert to SnapVault, just like in a DR-scenario - but it will take long, and in the meantime we would have no archive...

Any other suggestion?

thanks

Mark

aborzenkov
3,483 Views

It may be possible to use reallocate -p for physical reallocation ... but I may be it makes sense at least try to contact NetApp support for their comments.

In any case, if you will resolve this issue I appreciate if you post summary here. Thank you in advance!

mheimberg
3,483 Views

reallocate -p is doing only a scan as well - so I contacted NetApp support.

They told me, that not the secondary but the *primary* data has to be optimized.

So I ran I reallocate measure on the primary, with an index of 17 for optimization (in fact the client complainded for some months about poor performance...)

Nex I started a

>reallocate start -n -o /vol/SnapMgr_SQLServer_CHDB008_backup/db/iscsi_db008_e.lun

Unfortunately we had not enough free space in the volume nor the aggregate to support a full optimization, so I stopped the process after 25%, the snapshots increased in size by more than 350GB.

A new reallocate measure showed now

on the primary: optimization 13

on the secondary: optimization 8

We are checking now the performance.

Mark

mheimberg
3,483 Views

After running reallocation twice we have now

Primary: optimization index: 11

Secondary: optimization index: 5

The verify is taking about 4hrs less, that is about 11 hrs instead of 15.

So the issue is solved and we know have a means for optimizing runtimes.

Mark

Public