SnapMirror sync visibility interval confusion

[aborzenkov]

SM (semi-)sync documentation states, that although NVRAM/CP-sync fowarding happens immediately, client still sees data corresponding to the latest "visibility snapshot" which by default is done every 3 minutes. To naive reader it sounds, like in spite of "sync" I am going to lose up to 3 minutes data.

I could not find any description how data, that has been transferred since the latest visibility snapshot, can be made available on destination. Could somebody clarify it?

To make it clear, what I mean:

When replicating synchronously or semi-synchronously, using SnapMirror,  changes to the source volume do not show immediately on the destination  volume, even though the changes have been replicated. The changes are  shown only after the source system takes an automatic Snapshot  copy of the source volume.

And yes, I know about TR-3326, it does not answer this question, unfortunatly.

[scottgelb]

The visibility interval is for read only visibility, so you can only view up to the last 3 minutes, however the synchronous updates are on the target volume, just not visible.  If you create a snapshot on the source, that snapshot will be visible before the 3 minute interval.  If you break the mirror on the target volume, you will have all of the updates that were not visible (NVLOG, CPLOG are applied) so changing the visibility interval doesn't change the Recovery Point on disk in the event of a snapmirror break.  It only changes the read only view of the volume.

[aborzenkov]

That's what I would expect too, but as mentioned there is no clear statement anywhere in documentation which I have seen (training, DOT manuals, TR, various presentations on fieldportal). I'd love someone from NetApp to confirm it

[scottgelb · Accepted solution]

I have no doubt this is how it works...the interval controls creation of source snapshots to make them visible on the target but the sync update is available... break a mirror as a test and the data will be newer than the last visibility snapshot.  It would be nice if the documentation was more clear and hopefully an official response is listed.  I checked some old notes when snapmirror sync first came out and wrote down that visibility doesn't mean the state of the data on the target when you break the mirror.

[alapati]

Hi there,

Here is the excerpt from TR3326 that I authored. I believe the last part of the first sentence clarifies this - "even though the changes have been replicated."

Hope this helps,

Srinath

Hope this helps.

With SnapMirror Sync and SnapMirror Semi-Sync, changes to the source volume do not show up immediately on the destination volume, even though the changes have been replicated. The changes are first shown after the source system creates an automatic Snapshot copy of the source volume. Snapshot copies created on the source volume are immediately propagated to the destination volume. The automatic Snapshot copies are created every three minutes by default. To change the interval for automatic Snapshot copies, change the visibility_interval in the snapmirror.conf file; however, performance can degrade if set to a smaller interval because frequent Snapshot copies cause additional processing such as cache consistency checking on the destination system. There is also a disadvantage in setting this to a large value. When the connection between the systems has an outage and SnapMirror goes into asynchronous mode, SnapMirror uses the last common Snapshot copy to transition into synchronous mode. This means that all data from the last valid common Snapshot copy will need to be replicated from the source to the destination storage system. If the visibility_interval is set to a large value, a large amount of data might have to be transferred and it might take longer time for SnapMirror to transition into synchronous mode. For these reasons, keep the default value of three minutes.

[joubert]

Hi Srinath,

I've been reading TR-3326 several times, especially the portions on how sync snapmirror works.  As I understand, NVLOG forwarding sends the write operations to the target controller.  This is used to rebuild any previous transactions in case of a failure.  Then, there is CP Forwarding, which sends the data blocks from the source to the target when a CP a triggered, either after 10 seconds or if the NVRAM is full.  Thirdly, every 3 minutes a snapshot is created at the source, and then this snapshot copy is sent to the destination volume.

I'm okay with the NVLOG forwarding and CP Forwarding, but since we are already sending the data blocks when a CP is triggered which the target system already writes to disk, why do we still need to create and send snapshot copies from the source to the destination?  Are we sending the same data over the wire again, or are we just instructing the destination controller to take a snapshot as well?

Please help clarify.

Thanks!

Joubert

[mwalters]

Any snapshot info will only be metadata, nothing onerous 🙂

[tyrone_owen_1]

I'm just trying to understand SnapMirror Sync and I'm stumped by the relevance of the snapshots post initial baseline transfer and TR-3326 doesn't really give enough detail. There's no mention of them in section 3.2 'HOW SNAPMIRROR SYNC WORKS'. Can someone knit in an explanation amongst the TR's outline below pls?

After the baseline transfer is complete, SnapMirror can transition into synchronous mode:

1. A first asynchronous SnapMirror update occurs as described earlier
2. Consistency point (CP) forwarding begins. This is a method to make sure that writes of data from memory to disk storage are transferred from the primary system to the secondary system. For more information, see the CP forwarding section later in this guide. A second asynchronous SnapMirror update is started.
3. After the second asynchronous SnapMirror update completes, NVLOG forwarding begins. This is a method to transfer updates as they occur. For more information, see the NVLOG forwarding section later in this guide.
4. New writes from clients or hosts to the primary file system begin to block until acknowledgment of those writes has been received from the secondary system.

Thanks

[JOHNSONCHRIS3]

Hi joubert,

I know this thread is getting pretty dated but, to my understanding, in response to your question as to "why do we still need to create and send snapshot copies from the source to the destination?  Are we sending the same data over the wire again, or are we just instructing the destination controller to take a snapshot as well?", it is for VISIBILITY only.  This will allow you to look at data that has transferred over the line in 3 minute intervals (by default).  The snapshot is created on the source and propagated to the destination immediately.  The snapshot metadata combined with the actual data will provide visibility (on the destination) as to which data has transferred at the time of the snapshot.

Hope this helps,

Chris