Access denied when running robocopy the second time to update changed files

[kennethostnes]

Hi,

we are migrating CIFS users files( profiles, homefolders etc) from Windows 2003 Clusters to Netapp Vfilers. Robocopy is one of the tools used for migration.

Robocopy options used:

robocopy Z:\files\ \\NetappVfiler\Users\files\ /E /SEC /COPYALL /ZB /R:0 /W:0/mir /np /xd "RECYCLER" "System Volume Information" "Thumbs.db" /log:robocopy1.log

The first copy runs fine. All catalogs/files and their security settings are the same on destination as the source.  
Also on files that is not owned by the admin user logged in when we use Robocopy. Everything is just fine.

But when we try to run Robocopy the second time we got access denied on files that is not owned by the admin user.

robocopy Z:\files\ \\NetappVfiler\Users\fles\ /E /SEC /COPYALL /ZB /R:0 /W:0 /mir /np /xd "RECYCLER" "System Volume Information" "Thumbs.db" /log:robocopy1.log

Newer  10        New Text Document.txt

2011/11/16 13:13:02 ERROR 5 (0x00000005) Copying File Z:\files\New Text Document.txt

Access is denied.

Running a sectrace shows:

/files/New Text Document.txt

Wed Nov 16 13:13:03 CET [NetappVfiler@NetApp-V3240: sectrace.filter.allowed:info]: [vfiler: NetappVfiler: 3] Access allowed because the file is being opened for backup - Status: 1:142671872:0:0 - 172.17.12.24 - NT user name: \administrator - UNIX user name: pcuser(65534) - Qtree security style is NTFS and NT ACL is set on file/directory - Path: /vol/qt1/files/New Text Document.txt

Wed Nov 16 13:13:03 CET [NetappVfiler@NetApp-V3240: sectrace.filter.denied:info]: [vfiler: NetappVfiler sectrace index: 2] Access denied because 'Write' permission (0x2) is not granted on file or directory (Access denied because the requested permissions are not granted by the access control entries) - Status: 1:239075332:32:61 - 172.17.12.24 - NT user name: \administrator - UNIX user name: pcuser(65534) - Qtree security style is NTFS and NT ACL is set on file/directory - Path: /vol/qt1

NetApp-V3240> vfiler run NetappVfiler sectrace print-status 1:239075332:32:61

=====NetappVfiler

Access denied because requested permission is not granted on file or directory.

- Access allowed by share-level ACL.

- Access denied because the requested permissions are not granted by the access control entries.

NetApp-V3240>

Yes we know that we do not have the access to the file. The user we are using is an administrator in the windows domain and on the Vfiler. And is a member of the local Backup Operator on the Netapp Vfiler.

If we do a robocopy between two windows servers the job is run's fine on the same files, no access denied.

Do we have a problem with the backup opertator modus on Netapp. Is there a CIFS "Superuser" on Netapp we should be using?

[bobpowell]

Hi There

I think you may have a similar problem to one i encounter recently.

After upgrading our Filers to Ontap 8 and 64bit volumes our robocopy copies from windows file servers (32bit) stopped coping files complaining of NTFS security permissions. After investigation we found issues relating to Robocoy and 64bit OS's.

Well after a bit of trial and error the following worked for us, i think looking at your command there is too many switches.

Give it a try and see if it works out for you too.

robocopy d:\userarea \\netappfiler\userarea /s /e /zb /SEC /purge /r:3 /w:1 /v /eta /log+:\\netappfiler\logs\robolog.log /tee

All the best

[chinchilla]

When doing similar migrations, first robocopy we do without the /sec, and the second (or last) with it. In our case, it helps us to avoid the issues with folders which even admin accounts don't have access to (created by another domain user and having only other domain permissions that we are not entitled to change). Even for large number of files, security is applied in feasible time during the last copy.

[BONA_BONE1]

Hello,

Did you manage to get around this? Having same problem!?

Thanks

[kennethostnes]

Hi Paul,

Yes, the customer with this problem managed to find a workaround. Using a combination of different tools.

Regards

Kenneth

[BONA_BONE1]

Thanks, Don't suppose you know which tools, I am migrating roaming profiles using robocopy, the first "run" went fine, but now anything new / deleted is not being mirrored across to the destination, just get access denied with anything new or newer

Paul

[kennethostnes · Accepted solution]

Hi again,

They used a product called copyright I think. And they got a new fix build 093 of the program.

http://www.sys-manage.com/PRODUCTS/CopyRight/tabid/64/Default.aspx

From the customers e-mails

"I had to introduce a new commandline switch /NetappWorkaround.

I installed build 93 on the server that I was working on. It is located in "c:\program files (x86)\sys-manage\copyright\copyright.exe".

I updated the NetappTest demo so we could forward it to NetApp and it is now getting clearer what is going wrong:

1. The first attempt to create the destination file succeeds. It is created by using the CreateFile API using the parameter CREATE_ALWAYS. The file does not exist at this point and is created correctly.

2. Any subsequent runs will attempt to open the (already existing) destination file again with CREATE_ALWAYS. This fails on Netapp under the following conditions (the file is not owned by administrators but instead by a user account and the administrators do not have permissions to the file). However, opening the file with OPEN_EXISTING succeeds on Netapp. On Windows it does not matter wether you specify CREATE_ALWAYS or OPEN_EXISTING. The work around now uses OPEN_EXISTING in case the file is already there and then calls SetEndOfFile to truncate the file to zero bytes. "

Regards

Kenneth

[has000123]

Hi!

I have the same problems with robocopy and ONTAP 8.1.2. It works OK with Windows 2008R2. This has to be a long known bug in ONTAP will this be fixed?

Regards, Harald.

[ippccunningham]

I just ran into this exact same problem while transferring data from a Window 2008 (non R2) server.  It turned out to have nothing to do with the server I was copying from nor was there a bug in OnTap.  The problem I had was the qtree security for the volume was set to UNIX.  Once I changed it to NTFS, the files began to copy without error.

[andy1234567]

I am the one that debugged the problem. Our software CopyRight, just like robocopy, showed an error message (access denied) during the copy progress when running a second time and stopped. I did write a very small C program to isolate the problem and to seperate it from the product. This test program behaved differently depending on wether it ran against a Windows server or against a NetApp filer. It used the CreateFile API to create a file with the CREATE_ALWAYS flag set, which means to create the file if it is not there or overwrite it and truncate it to 0 bytes if the file is already existent. However, during the first run the file was always created on both systems (Windows and NetApp), during any subsequent runs, with the file to be created already in place, the two systems behaved differently. On Windows it worked as expected without any error, but on NetApp it returned an access denied error when calling CreateFile. I think this error is/was either an OnTAP bug or a bug in the Samba implementation used. The solution/workaround for NetApp was to open the file with OPEN_EXISTING instead of CREATE_ALWAYS, which suprisingly did not cause the access denied error and then truncate the file to 0-bytes by using an additional API call to TruncateFile. This fixed the issue for our customer and he migrated successfully to NetApp. We have implemented the workaround in our product.