Network and Storage Protocols

Permission denied on NTFS volume shared by NFS

sassynatan
10,519 Views

Hi All,

Well the question is easy, but the answer is hard 🙂

I have in my infrastcture 10 Linux machine running RHEL 5.0.

The 10 Linux machines are powers boxes (each with 24 cores and 96GB of RAM)

All my users are loging to these machine using NX client.

The Linux machine are configure to work with the Microsoft Active Driecroty servivces.

So all users infromation coming from the LDAP (UID, GID, gecons etc....)

Authentication is done by kerberos.

I also configure the LDAP options on the NetApp and it seems to work fine (getXXbyYY works!)

Anyway, this setup was working great for the last 3 years. No issues at all!

The problem is with RHEL 6.0!

This version was installed, and used the same setup as in version 5.0.

Authentication is done by kerberos, and I used the nslcd option to get info from the ldap server.

So far everything is good.

I also manage to mount the NFS shares on the machine.

But the problem I now facing is that on NFS share, which is a NTFS qtree based I get permssion denied when trying to copy a file.

Here is a copy paste:

RHEL6.0:

[sassy.natan@cpu06 /]# mount

/dev/sda1 on / type ext4 (rw)

proc on /proc type proc (rw)

sysfs on /sys type sysfs (rw)

devpts on /dev/pts type devpts (rw,gid=5,mode=620)

tmpfs on /dev/shm type tmpfs (rw)

none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)

/tmp on /tmp type none (rw,bind)

/var/tmp on /var/tmp type none (rw,bind)

/home on /home type none (rw,bind)

sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)

filer:/vol/Users on /home type nfs (rw,addr=172.16.0.6)                                                                This is a UNIX QTREE!

filer:/vol/Softwares on /mnt/Softwares type nfs (rw,addr=172.16.0.6)

filer:/vol/Common on /mnt/Common type nfs (rw,addr=172.16.0.6)

filer:/vol/Groups on /mnt/Groups type nfs (rw,addr=172.16.0.6) -                                                   This is the NTFS QTREE!

AppSrv:/var/www/soconline/soconline on /mnt/Soconline type nfs (rw,addr=172.16.0.5)

[sassy.natan@cpu06]/home/sassy.natan(85): cd /mnt/Groups/IT                                                  NTFS Directory where user sassy.natan as a full permssion!

[sassy.natan@cpu06]/mnt/Groups/IT(86): ll

total 9108

drwx------  9 root          root            4096 Aug 15 16:12 .

dr-xr-xr-x 21 root          root            4096 Jun 14 12:33 ..

drwx------  2 azhar.sharoof Domain Users    4096 Jan  1  2009 2008

drwx------  3 azhar.sharoof Domain Users    4096 Aug  1 09:46 2009

drwx------  2 azhar.sharoof Domain Users    4096 Jun 21 19:33 2010

drwx------  2         10041 Domain Users    4096 Mar 14 15:44 :DA96000

drwx------ 12 root          bin             4096 Mar  2 14:32 How To

drwx------  8         10041 Domain Users    4096 Nov  3  2008 Licence

drwx------  2 root          bin             4096 Nov 10  2009 Operations

-rwx------  1 sassy.natan   Domain Users 9240576 Aug 15 16:11 output.file

-rwx------  1 azhar.sharoof Domain Users   11431 Feb  9  2011 Printer Mapping.xlsx

-rwx------  1 root          bin            16013 Sep 28  2010 Telephony.xlsx

[sassy.natan@cpu06]/mnt/Groups/IT(87): touch file                                                                          Here I touch a simple file

[sassy.natan@cpu06]/mnt/Groups/IT(88): ls -la

total 9108

drwx------  9 root          root            4096 Aug 15 16:58 .

dr-xr-xr-x 21 root          root            4096 Jun 14 12:33 ..

drwx------  2 azhar.sharoof Domain Users    4096 Jan  1  2009 2008

drwx------  3 azhar.sharoof Domain Users    4096 Aug  1 09:46 2009

drwx------  2 azhar.sharoof Domain Users    4096 Jun 21 19:33 2010

drwx------  2         10041 Domain Users    4096 Mar 14 15:44 :DA96000

-rwx------  1 sassy.natan   Domain Users       0 Aug 15 16:58 file                                                       and here it is 🙂

drwx------ 12 root          bin             4096 Mar  2 14:32 How To

drwx------  8         10041 Domain Users    4096 Nov  3  2008 Licence

drwx------  2 root          bin             4096 Nov 10  2009 Operations

-rwx------  1 sassy.natan   Domain Users 9240576 Aug 15 16:11 output.file

-rwx------  1 azhar.sharoof Domain Users   11431 Feb  9  2011 Printer Mapping.xlsx

-rwx------  1 root          bin            16013 Sep 28  2010 Telephony.xlsx

[sassy.natan@cpu06]/mnt/Groups/IT(89): mkdir test                                                                        here I create a dir

[sassy.natan@cpu06]/mnt/Groups/IT(91): ls -la

total 9112

drwx------ 10 root          root            4096 Aug 15 16:59 .

dr-xr-xr-x 21 root          root            4096 Jun 14 12:33 ..

drwx------  2 azhar.sharoof Domain Users    4096 Jan  1  2009 2008

drwx------  3 azhar.sharoof Domain Users    4096 Aug  1 09:46 2009

drwx------  2 azhar.sharoof Domain Users    4096 Jun 21 19:33 2010

drwx------  2         10041 Domain Users    4096 Mar 14 15:44 :DA96000

-rwx------  1 sassy.natan   Domain Users       0 Aug 15 16:58 file

drwx------ 12 root          bin             4096 Mar  2 14:32 How To

drwx------  8         10041 Domain Users    4096 Nov  3  2008 Licence

drwx------  2 root          bin             4096 Nov 10  2009 Operations

-rwx------  1 sassy.natan   Domain Users 9240576 Aug 15 16:11 output.file

-rwx------  1 azhar.sharoof Domain Users   11431 Feb  9  2011 Printer Mapping.xlsx

-rwx------  1 root          bin            16013 Sep 28  2010 Telephony.xlsx

drwx------  2 sassy.natan   Domain Users    4096 Aug 15 16:59 test                                                     and here it is 🙂

And now the problem:

Here I going to copy file output.file to the same folder

[sassy.natan@cpu06]/mnt/Groups/IT(92): cp -rf output.file output.file.test

cp: cannot create regular file `output.file.test': Permission denied

Exit 1

Doing ls - seems to create the file in size 0:

[sassy.natan@cpu06]/mnt/Groups/IT(93): ls -la

total 9112

drwx------ 10 root          root            4096 Aug 15 17:02 .

dr-xr-xr-x 21 root          root            4096 Jun 14 12:33 ..

drwx------  2 azhar.sharoof Domain Users    4096 Jan  1  2009 2008

drwx------  3 azhar.sharoof Domain Users    4096 Aug  1 09:46 2009

drwx------  2 azhar.sharoof Domain Users    4096 Jun 21 19:33 2010

drwx------  2         10041 Domain Users    4096 Mar 14 15:44 :DA96000

-rwx------  1 sassy.natan   Domain Users       0 Aug 15 16:58 file

drwx------ 12 root          bin             4096 Mar  2 14:32 How To

drwx------  8         10041 Domain Users    4096 Nov  3  2008 Licence

drwx------  2 root          bin             4096 Nov 10  2009 Operations

-rwx------  1 sassy.natan   Domain Users 9240576 Aug 15 16:11 output.file

-rwx------  1 sassy.natan   Domain Users       0 Aug 15 17:02 output.file.test               file size is 0

-rwx------  1 azhar.sharoof Domain Users   11431 Feb  9  2011 Printer Mapping.xlsx

-rwx------  1 root          bin            16013 Sep 28  2010 Telephony.xlsx

drwx------  2 sassy.natan   Domain Users    4096 Aug 15 16:59 test

Doing again copy now seems to work:

[sassy.natan@cpu06]/mnt/Groups/IT(94):  cp -rf output.file output.file.test

cp: overwrite `output.file.test'? y

[sassy.natan@cpu06]/mnt/Groups/IT(95): ll

total 18156

drwx------ 10 root          root            4096 Aug 15 17:02 .

dr-xr-xr-x 21 root          root            4096 Jun 14 12:33 ..

drwx------  2 azhar.sharoof Domain Users    4096 Jan  1  2009 2008

drwx------  3 azhar.sharoof Domain Users    4096 Aug  1 09:46 2009

drwx------  2 azhar.sharoof Domain Users    4096 Jun 21 19:33 2010

drwx------  2         10041 Domain Users    4096 Mar 14 15:44 :DA96000

-rwx------  1 sassy.natan   Domain Users       0 Aug 15 16:58 file

drwx------ 12 root          bin             4096 Mar  2 14:32 How To

drwx------  8         10041 Domain Users    4096 Nov  3  2008 Licence

drwx------  2 root          bin             4096 Nov 10  2009 Operations

-rwx------  1 sassy.natan   Domain Users 9240576 Aug 15 16:11 output.file

-rwx------  1 sassy.natan   Domain Users 9240576 Aug 15 17:03 output.file.test               FILE COPIED!

-rwx------  1 azhar.sharoof Domain Users   11431 Feb  9  2011 Printer Mapping.xlsx

-rwx------  1 root          bin            16013 Sep 28  2010 Telephony.xlsx

drwx------  2 sassy.natan   Domain Users    4096 Aug 15 16:59 test

[sassy.natan@cpu06]/mnt/Groups/IT(96):

In the RHEL 5.0 this works without a problem 🙂

Here is a copy of the /proc/mounts on RHEL 6.0 vs RHEL 5.0

RHEL 6.0:

filer:/vol/Users /home nfs rw,relatime,vers=3,rsize=65536,wsize=65536,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=172.16.0.6,mountvers=3,mountport=4046,mountproto=udp,addr=172.16.0.6 0 0

filer:/vol/Softwares /mnt/Softwares nfs rw,relatime,vers=3,rsize=65536,wsize=65536,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=172.16.0.6,mountvers=3,mountport=4046,mountproto=udp,addr=172.16.0.6 0 0

filer:/vol/Common /mnt/Common nfs rw,relatime,vers=3,rsize=65536,wsize=65536,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=172.16.0.6,mountvers=3,mountport=4046,mountproto=udp,addr=172.16.0.6 0 0

filer:/vol/Groups /mnt/Groups nfs rw,relatime,vers=3,rsize=65536,wsize=65536,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=172.16.0.6,mountvers=3,mountport=4046,mountproto=udp,addr=172.16.0.6 0 0

RHEL 5.0:

filer:/vol/Users /home nfs rw,vers=3,rsize=65536,wsize=65536,hard,proto=tcp,timeo=600,retrans=2,sec=sys,addr=filer 0 0

filer:/vol/Softwares /mnt/Softwares nfs rw,vers=3,rsize=65536,wsize=65536,hard,proto=tcp,timeo=600,retrans=2,sec=sys,addr=filer 0 0

filer:/vol/Common /mnt/Common nfs rw,vers=3,rsize=65536,wsize=65536,hard,proto=tcp,timeo=600,retrans=2,sec=sys,addr=filer 0 0

filer:/vol/Groups /mnt/Groups nfs rw,vers=3,rsize=65536,wsize=65536,hard,proto=tcp,timeo=600,retrans=2,sec=sys,addr=filer 0 0

/etc/fstab on both server looks like this:

filer:/vol/Users                        /home                   nfs     defaults        0 0

filer:/vol/Softwares                    /mnt/Softwares          nfs     defaults        0 0

filer:/vol/Common                       /mnt/Common             nfs     defaults        0 0

filer:/vol/Groups                       /mnt/Groups             nfs     defaults        0 0

Any ideas?

Thanks

Sassy

4 REPLIES 4

sassynatan
10,519 Views

FYI, I still don't know what is the different, but moving to NFSv4 on RHEL 6.0 solved the problem.

I do however think this is a bug or some configuration error, and will be happy if someone can tell what is the cause of it.

As I see it there no reason why this shouldn't working the same as in version 5.0 of RedHat.

Thanks

Sassy

aborzenkov
10,519 Views

Start with stracing cp command to find out what system call exactly returns EPERM. I suspect it is attempt to (explicitly) change file mask/permissions, because file is obviously created.

thomasnetzker
10,519 Views

I've experienced exactly the same problem: its happen only with RHEL6, NFS v3 and qtree sec-style NTFS. It first creates an empty file while saying permission denied and fills the content of the file on the second copy. Strace showed the following while trying to copy ADR.flt for the first time:

..

f_files=2485504, f_ffree=2388770, f_fsid={2063430184, 312728360}, f_namelen=255, f_frsize=4096}) = 0

brk(0)                                  = 0x1da3000

brk(0x1dc4000)                          = 0x1dc4000

open("/proc/filesystems", O_RDONLY)     = 3

fstat(3, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0

mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7ff61c995000

read(3, "nodev\tsysfs\nnodev\trootfs\nnodev\tb"..., 1024) = 342

read(3, "", 1024)                       = 0

close(3)                                = 0

munmap(0x7ff61c995000, 4096)            = 0

open("/usr/lib/locale/locale-archive", O_RDONLY) = 3

fstat(3, {st_mode=S_IFREG|0644, st_size=99158752, ...}) = 0

mmap(NULL, 99158752, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7ff616af2000

close(3)                                = 0

open("/proc/filesystems", O_RDONLY)     = 3

fstat(3, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0

mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7ff61c995000

read(3, "nodev\tsysfs\nnodev\trootfs\nnodev\tb"..., 1024) = 342

read(3, "", 1024)                       = 0

close(3)                                = 0

munmap(0x7ff61c995000, 4096)            = 0

geteuid()                               = 2693

stat("./", {st_mode=S_IFDIR|0777, st_size=4096, ...}) = 0

stat("/dxm/dxmakt/dbsave/110718/ADR.flt", {st_mode=S_IFREG|0600, st_size=558830, ...}) = 0

stat("./ADR.flt", 0x7fff961c8990)       = -1 ENOENT (No such file or directory)

open("/dxm/dxmakt/dbsave/110718/ADR.flt", O_RDONLY) = 3

fstat(3, {st_mode=S_IFREG|0600, st_size=558830, ...}) = 0

open("./ADR.flt", O_WRONLY|O_CREAT|O_EXCL, 0600) = -1 EACCES (Permission denied)

open("/usr/share/locale/locale.alias", O_RDONLY) = 4

fstat(4, {st_mode=S_IFREG|0644, st_size=2512, ...}) = 0

mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7ff61c995000

read(4, "# Locale name alias data base.\n#"..., 4096) = 2512

read(4, "", 4096)                       = 0

close(4)                                = 0

munmap(0x7ff61c995000, 4096)            = 0

open("/usr/share/locale/en_US.UTF-8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)

open("/usr/share/locale/en_US.utf8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)

open("/usr/share/locale/en_US/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)

open("/usr/share/locale/en.UTF-8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)

open("/usr/share/locale/en.utf8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)

open("/usr/share/locale/en/LC_MESSAGES/coreutils.mo", O_RDONLY) = 4

fstat(4, {st_mode=S_IFREG|0644, st_size=435, ...}) = 0

mmap(NULL, 435, PROT_READ, MAP_PRIVATE, 4, 0) = 0x7ff61c995000

close(4)                                = 0

write(2, "cp: ", 4)                     = 4

write(2, "cannot create regular file `./AD"..., 38) = 38

open("/usr/share/locale/en_US.UTF-8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)

open("/usr/share/locale/en_US.utf8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)

open("/usr/share/locale/en_US/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)

open("/usr/share/locale/en.UTF-8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)

open("/usr/share/locale/en.utf8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)

open("/usr/share/locale/en/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)

write(2, ": Permission denied", 19)     = 19

write(2, "\n", 1)                       = 1

close(3)                                = 0

close(0)                                = 0

close(1)                                = 0

close(2)                                = 0

exit_group(1)                           = ?

gopinathp
10,519 Views

We had the similar issue with some of our Linux clients. One solution was to set this hidden option on the affected filers:

filer>options cifs.ntfs_ignore_unix_security_ops on

For more info

https://kb.netapp.com/support/index?page=content&id=3011859

Public