Is volume overwrite rate a good indicator for rate of change when calculating NDMP incremental backup workload

[RichardSopp]

I'm assuming that the overwrite rate will report at the block level but NDMP operates at the file level.  I was wondering if there was a better way of calculating this in terms of NDMP planning?

Also does the Volume Overwrite Rate report on Operations Manager report the overwrite rate in terms of the rate between the last sample and the current sample, or does it average the rate out over history?  If an average over what period is the average derived?

Any input would be appreciated.

[adaikkap · Accepted solution]

Hi richard,

     I am not complete sure on the answer for the first question.

But i can give you the answer for how overwrite is calculated.

SECONDS_PER_DAY = 60 * 60 * 24;
    BYTES_PER_KB = 1024;

    overwritten_bytes_per_sec = "consumed-size" / "elapsed-time";
    
    overwritten_kilobytes_per_day = ((overwritten_bytes_per_sec * SECONDS_PER_DAY)
                                / BYTES_PER_KB);

Where 
consumed-size is Size in bytes of space changed between the the oldest and latest snapshots.
elapsed-time is Time in seconds elapsed between the 2 specified snapshots

Regards
adai

[smoot]

Hi Richard --

Like Adai said, the overwrite rate is the long-term overwrite rate, computed from the oldest snapshot to the active file system.  It's not a bad estimate but I think it will tend to underestimate (in case the same block keeps getting overwritten, you'll need to dump it to tape every day, but it won't show up in the long-term overwrite rate).

To get the best estimate, you'd want to run "snap delta" on successive pairs of daily snapshots and track how that changes over time.  I don't think DFM has any reports which do this, but it sounds like an interesting enhancement request.  We have to be careful how we implement this because running "snap delta" is expensive on the storage system.

-- Pete

[RichardSopp]

To get the best estimate, you'd want to run "snap delta" on successive pairs of daily snapshots and track how that changes over time.  I don't think DFM has any reports which do this, but it sounds like an interesting enhancement request.  We have to be careful how we implement this because running "snap delta" is expensive on the storage system.

Thanks Pete and Adai.

I thought OM already used the snap delta API to present the overwrite rate in this report.  Am I wrong?

Pete - your answer brings up an interesting question regarding a use case where a storage system may not have successive paris of daily snapshots, either because only weeklys are defined or your are potentially snapvaulting them to a separate system.   Is the report specifically looking for dailys or will it work on any successive collections of snap irrespective of their relationship?

[smoot]

Hi Richard --

No, you're exactly correct.  One of the OpsManager monitors runs "snap delta" to compute the number of changed bytes between the oldest snapshot and the active file system.  Divide by the age of the oldest snapshot and you've got the long-term overwrite rate.

We don't look for matching daily or weekly snapshots for exactly the reasons you point out: it's difficult to determine which ones to use and awkward to figure out what to do when we don't find the right pair.  There's always an oldest snapshot.

The enhancement I was thinking of would be to somehow enable you to tell us which snapshot pairs are most relevent to you so we could measure the overwrite rate which matters in your environment.  We'd have to figure out how to deal with the error cases if we implemented that.

-- Pete

[eric_barlier]

hi guys,

in OM under "member details" and "File Systems" tab select <snapshot details>. You should be able to select a few volumes and at bottom of page you can push: snap reclaimable.

that would help you out maybe? Be aware that if you run IE 6 there is a limit of 126 snaps at one time you can select..

Cheers,
Eric