Network and Storage Protocols

vfiler dr resync failing

PKKASTURI

Hi,

I'm trying to perform vfiler dr resync from one filer to other filer. But didn't success, getting below error. I initially did vfiler dr configure on destination filer (netapp-02) and it got successfull. i've added some data and now i'm trying to syncing the data from destination (netapp-02) filer to source (netapp-01) filer.

netapp-01> vfiler status

vfiler0                          running

vf-test                          stopped

netapp-01> vfiler dr resync vf-test@netapp-02

netapp-02's Administrative login: root

netapp-02's Administrative password:

cannot resync path /vol/src_vol: transfer aborted because of network error

Mon Aug 27 19:55:54 GMT [replication.dst.resync.failed:error]: SnapMirror resync of src_vol to netapp-02:src_vol : transfer aborted because of network error.

netapp-01> df -h test_vol

Filesystem               total       used      avail capacity  Mounted on

/vol/test_vol/           240MB       66MB      173MB      28%  /vol/test_vol/

netapp-01> snapmirror status

Snapmirror is on.

Source                Destination           State          Lag        Status

netapp-01:test_vol    netapp-02:sm_vol      Source         -          Idle

netapp-01:src_vol     netapp-02:src_vol     Source         161:51:16  Idle

netapp-01:test_vol    netapp-02:test_vol    Source         161:51:14  Idle

netapp-01>

netapp-02> df -h test_vol

Filesystem               total       used      avail capacity  Mounted on

/vol/test_vol/           240MB       73MB      166MB      31%  /vol/test_vol/

netapp-02> snapmirror status

Snapmirror is on.

Source                Destination           State          Lag        Status

netapp-01:src_vol     netapp-02:src_vol     Broken-off     06:25:12   Idle

netapp-01:test_vol    netapp-02:test_vol    Broken-off     06:25:10   Idle

netapp-02:src_vol     netapp-01:src_vol     Source         -          Idle

netapp-02:test_vol    netapp-01:test_vol    Source         -          Idle

netapp-02>

can anyone tell me what i'm missing?

Thanks,

Phani

23 REPLIES 23

spwilkinson

I'm having the same issue here when doing a vfiler dr resync. I'm using a sim running 8.2 for testing.

sim04> vfiler dr resync -c secure dr@sim03

sim03's Administrative login: root

sim03's Administrative password:

cannot resync path /vol/dr2_root: transfer aborted because of network error

sim04> Fri Mar 28 15:21:01 GMT [sim04:replication.dst.resync.failed:error]: SnapMirror resync of dr2_root to sim03:dr2_root : transfer aborted because of network error.

sim03> Fri Mar 28 15:21:01 GMT [sim03:snapmirror.src.resync.toSelf:error]: SnapMirror cannot resync volume dr2_root to itself, operation not permitted.

Eric_Johnson

I was having EXACTLY the same problem but was able to resolve it.  It had to do with the fact that the simulators all share a common sys_serial_num and sysid which was causing the resync to throw an error about syncing to itself.

 

I followed the procedure to change the System ID and ran through the setup again and it worked.

This is on Netapp's site but here's a c&p

 

*****************************************************

Setting the System ID
---------------------

IMPORTANT: You must enter the two following commands BEFORE the Simulate ONTAP node boots up for the first time.

Follow the steps below to change the System ID and System Serial Number for the second node:

1. Once you start the Simulate ONTAP node, the system will begin the boot up process in 10 seconds unless you interrupt it. You will see the following information on the console:

Hit [Enter] to boot immediately, or any other key for command prompt.
Booting in 10 seconds...

2. Press any key other than the Enter key to interrupt the boot process. You will see the following information:

Type '?' for a list of commands, 'help' for more detailed help.
SIMLOADER>

3. Enter the following commands to set the Serial Number and System ID for this node:

SIMLOADER> setenv SYS_SERIAL_NUM 4034389-06-2
SIMLOADER> setenv bootarg.nvram.sysid 4034389062

4. Enter the following commands to verify that the information was saved correctly:

SIMLOADER> printenv SYS_SERIAL_NUM
4034389-06-2
SIMLOADER> printenv bootarg.nvram.sysid
4034389062

5. Now enter the "boot" command to boot up the node:

SIMLOADER> boot

*****************************************************

 

 

netappz

Thanks you Eric - that information was great and would have sorted the issue I came across, except it broke my SIM as it had already been booted 🙂

 

Not to worry though, as least I now know why it was failing (the duplicate sysids - never would have figured that otherwise).

 

Thanks again.

scottgelb

Does a manual snapmirror work? Probably a mirror or network issue. If manual mirror fails then we can troubleshoot from there. If it works then a vFiler dr issue to look at.

Sent from my iPhone 5

spwilkinson

A manual snapmirror doesn't work either. The above output was from a sim running 8.1.4 not 8.2. I was just testing to make sure it wasn't an issue with 8.2 but it's exactly the same. Full output below from the 8.2 sim.

sim02> vfiler dr activate cifsvfiler@sim01

CIFS local server is running.

Fri Mar 28 17:43:58 GMT [cifsvfiler@sim02:cifs.startup.local.succeeded:info]: CIFS: CIFS local server is running.

Fri Mar 28 17:43:58 GMT [sim02:httpd.config.mime.missing:warning]: /etc/httpd.mimetypes.sample file is missing.

Fri Mar 28 17:43:58 GMT [cifsvfiler@sim02:httpd.config.mime.missing:warning]: /etc/httpd.mimetypes file is missing.

Fri Mar 28 17:43:58 GMT [cifsvfiler@sim02:httpd.config.mime.missing:warning]: /etc/httpd.mimetypes.sample file is missing.

Vfiler cifsvfiler activated.

e0a: flags=0xe48867<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500

        inet 192.168.0.21 netmask 0xffffff00 broadcast 192.168.0.255

        ether 00:0c:29:0e:d8:74 (auto-1000t-fd-up) flowcontrol full

sim02> Fri Mar 28 17:43:59 GMT [sim02:cmds.vfiler.dr.activated:info]: Disaster recovery backup vFiler unit: 'cifsvfiler' of the vFiler unit at remote storage system: 'sim01' was activated.

sim02> ifconfig e0a

e0a: flags=0xe48867<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500

        inet 192.168.0.21 netmask 0xffffff00 broadcast 192.168.0.255

        inet 192.168.0.120 netmask 0xffffff00 broadcast 192.168.0.255

        ether 00:0c:29:0e:d8:74 (auto-1000t-fd-up) flowcontrol full

sim01> vfiler dr resync -c secure cifsvfiler@sim02

sim02's Administrative login: root

sim02's Administrative password:

cannot resync path /vol/cifsvfiler_root: transfer aborted because of network error

sim01> Fri Mar 28 17:58:44 GMT [sim01:replication.dst.resync.failed:error]: SnapMirror resync of cifsvfiler_root to sim02:cifsvfiler_root : transfer aborted because of network error.

[sim01:snapmirror.src.resync.toSelf:error]: SnapMirror cannot resync volume cifs to itself, operation not permitted.

scottgelb

Check the snapmirror.allow settings in case they got wacked… then if you can ping between source and target and vice-versa… then make sure the VM network didn’t move or disconnect anything.

spwilkinson

Yep, checked that and had it was still set to *. I've changed it to the host and can ping between both sims by ip/hostname no problem.

sim02> options snapmirror

snapmirror.access            *

sim01> vfiler dr resync -c secure cifsvfiler@sim02

sim02's Administrative login: root

sim02's Administrative password:

cannot resync path /vol/cifsvfiler_root: transfer aborted because of network error

sim01> Fri Mar 28 18:34:03 GMT [sim01:replication.dst.resync.failed:error]: SnapMirror resync of cifsvfiler_root to sim02:cifsvfiler_root : transfer aborted because of network error.

sim02> Fri Mar 28 18:34:02 GMT [sim02:snapmirror.src.resync.toSelf:error]: SnapMirror cannot resync volume cifsvfiler_root to itself, operation not permitted.

scottgelb

Does snap list have a matching snapshot between the two volumes (probably does and doesn’t even seem to get that far but checking)… if you restrict the target volume does snapmirror manually work?

spwilkinson

Snap list below. The snapmirror relationship is sim01-sim02. Does the vfiler dr resync copy any new data back to the original source from the now active dr vfiler?

sim02>

Volume cifsvfiler_root

working...

  %/used       %/total  date          name

----------  ----------  ------------  --------

32% (32%)    0% ( 0%)  Mar 28 17:42  sim02(4079432748)_cifsvfiler_root.2

39% (15%)    0% ( 0%)  Mar 28 17:41  sim02(4079432748)_cifsvfiler_root.1

Volume cifsvfiler_data

working...

  %/used       %/total  date          name

----------  ----------  ------------  --------

36% (36%)    0% ( 0%)  Mar 28 17:42  sim02(4079432748)_cifsvfiler_data.2

54% (38%)    0% ( 0%)  Mar 28 17:41  sim02(4079432748)_cifsvfiler_data.1

Volume cifs

working...

  %/used       %/total  date          name

----------  ----------  ------------  --------

27% (27%)    0% ( 0%)  Mar 28 17:42  sim02(4079432748)_cifs.2

42% (27%)    0% ( 0%)  Mar 28 17:41  sim02(4079432748)_cifs.1

sim01>

Volume cifsvfiler_root

working...

  %/used       %/total  date          name

----------  ----------  ------------  --------

13% (13%)    0% ( 0%)  Mar 28 17:42  sim02(4079432748)_cifsvfiler_root.2 (snapmirror)

Volume cifsvfiler_data

working...

  %/used       %/total  date          name

----------  ----------  ------------  --------

34% (34%)    0% ( 0%)  Mar 28 17:42  sim02(4079432748)_cifsvfiler_data.2 (snapmirror)

Volume cifs

working...

  %/used       %/total  date          name

----------  ----------  ------------  --------

21% (21%)    0% ( 0%)  Mar 28 17:42  sim02(4079432748)_cifs.2 (snapmirror)

scottgelb

On the controller where you run dr resync the volumes revert back / snaprestore to the last common snapshot. So you would dr resync back to production. Stop dr. Activate production then dr resync back to dr.

You have common snaps below. Snapmirror resync should work. Not sure what the network error is. Can you create a new mirror going the same direction separate from the vfilers?

Sent from my iPhone 5

spwilkinson

Thought so. I can create a new snapmirror relationship and transfer no problem. There is the error below on the source filer from the dr resync.

Transfer Status: resync to self not allowed

scottgelb

It looks like it thinks the mirror is the same volume source and target. Have not seen dr resync do this. Anything weird or different in snapmirror.conf or dns or hosts files pointing to the same controller?

Sent from my iPhone 5

SHYAMJAIK

Thank you Scott for the tip. Phani, Connectivity checks were all done to the source filer? also DNS, /etc/hosts.?

SHYAMJAIK

Can you check the value in snapmirror.allow file in etc on the source filer?

scottgelb

Access is set to all so allow file is not used. Only when set to legacy. We are all thinking similarly which is good

Sent from my iPhone 4S

andrewgeorge

try doing a snapmirror resyc of the individual volume (and if that works break it again)

I'd be suspecting that src_vol on netapp-01 has got a snapshot that can't be deleted automatically, so a snap list of src_vol on each filer would show the difference.

PKKASTURI

Hi,

Tried both vfiler dr resync and snapmirror resync commands, getting the same error mentioned below.

netapp-01>vfiler dr resync -c secure vf-test@netapp-02

netapp-02's Administrative login: root

netapp-02's Administrative password:

cannot resync path /vol/src_vol: transfer aborted because of network error

Mon Aug 27 22:33:22 GMT [replication.dst.resync.failed:error]: SnapMirror resync of src_vol to netapp-02:src_vol : transfer aborted because of network error.

netapp-01> snapmirror resync -S netapp-02:test_vol netapp-01:test_vol

Snapmirror resynchronization of test_vol to netapp-02:test_vol : transfer aborted because of network error

Aborting resync.

Mon Aug 27 22:32:11 GMT [replication.dst.resync.failed:error]: SnapMirror resync of test_vol to netapp-02:test_vol : transfer aborted because of network error.

the following are the log messges taken from snapmirror.log file.

Tue Aug 21 10:46:40 GMT [snapmirror.src.resync.toSelf:error]: SnapMirror cannot resync volume test_vol to itself, operation not permitted.

Tue Aug 21 11:00:04 GMT [kern.uptime.filer:info]:  11:00am up 22:45 0 NFS ops, 775 CIFS ops, 0 HTTP ops, 0 FCP ops, 0 iSCSI ops

Tue Aug 21 11:03:46 GMT [snapmirror.src.resync.toSelf:error]: SnapMirror cannot resync volume src_vol to itself, operation not permitted.

Tue Aug 21 11:04:23 GMT [snapmirror.src.resync.toSelf:error]: SnapMirror cannot resync volume test_vol to itself, operation not permitted.

Tue Aug 21 11:05:35 GMT [snapmirror.src.resync.toSelf:error]: SnapMirror cannot resync volume src_vol to itself, operation not permitted.

scottgelb

options snapmirror.access has permissions on both sides?  It may have been set up one way only..

PKKASTURI

No. kept snapmirror.access was set to "all" on both source and destination filers.

netapp-01> options snapmirror

snapmirror.access            all

netapp-02> options snapmirror

snapmirror.access            all

scottgelb

Snap list output from both controllers for that specific volume?

Sent from my iPhone 4S

Announcements
NetApp on Discord Image

We're on Discord, are you?

Live Chat, Watch Parties, and More!

Explore Banner

Meet Explore, NetApp’s digital sales platform

Engage digitally throughout the sales process, from product discovery to configuration, and handle all your post-purchase needs.

NetApp Insights to Action
I2A Banner
Public