topic Re: NFSv4 ACLs on RHEL? in ONTAP Discussions

NFSv4 ACLs on RHEL?

JOSHBAIRD — Thu, 05 Jun 2025 06:14:51 GMT

I'm having trouble using NFSv4 ACL's on RHEL6 from an exported volume with NFSv4+ACLs enabled.

On the client:

filer:/vol/vol4/share on /mnt/eportal type nfs4 (rw,rsize=65536,wsize=65536,hard,intr,proto=tcp,timeo=600,retrans=3,sec=sys,acl)

It is my understanding that I must use "nfs4_setfacl" on RHEL, because the POSIX enabled "setfacl" command does not work for NFSv4 ACLs. Whenever I try to use nfs4_setfacl to configure an ACL on a file/directory on the exported filesystem, I get the following error:

$ nfs4_setfacl -a A::jbaird@:rwatTnNcCy hi

Failed setxattr operation: Invalid argument

The documentation on this matter is very sparse, and I can't really find much. Can anyone offer some assistance?

Thanks!

Re: NFSv4 ACLs on RHEL?

JOSHBAIRD — Mon, 26 Nov 2012 15:13:32 GMT

Ok, this appears to be because the user that I am trying to configure the ACL with is a local user on the Linux system which is NOT on the Filer (in /etc/passwd, LDAP or NIS).

Re: NFSv4 ACLs on RHEL?

JOSHBAIRD — Mon, 26 Nov 2012 15:43:31 GMT

I still can't get ACL's to work with domain users (both the Filer and the Linux client have access to the same LDAP/AD directory). I get the "Failed setxattr operation" error. Anyone doing this?

Re: NFSv4 ACLs on RHEL?

parisi — Tue, 12 Mar 2013 19:42:59 GMT

So I know this is, like, 5 months after you wanted an answer, but in case you're still trying to get this to work, the issue is with the command syntax.

You did not specify your NFSv4 domain after your username.

When I run the command like you ran it, I get the same issue:

# nfs4_setfacl -a A::ldapuser@:rwatTnNcCy file

Failed setxattr operation: Invalid argument

When I run it with an nfsv4 domain specified, it works fine:

# nfs4_setfacl -a A::ldapuser@parisiwin2k3.netapp.com:rwatTnNcCy newkrb5 --test

## Test mode only - the resulting ACL for "/vfileralias/newkrb5":

A::ldapuser@parisiwin2k3.netapp.com:rwatTnNcCy

A::OWNER@:rwatTnNcCy

D::OWNER@:x

A:g:GROUP@:rtncy

D:g:GROUP@:waxTC

A::EVERYONE@:rtncy

D::EVERYONE@:waxTC

Re: NFSv4 ACLs on RHEL?

ccie5863 — Thu, 08 Aug 2013 14:59:13 GMT

Hello Parisi

we have still the same problem - with the same syntax as in your example.

# nfs4_setfacl -a A::ldapuser@parisiwin2k3.netapp.com:rwatTnNcCy newkrb5 --test

with the parameter --test - at the end of the command - everything looks ok !

but without the --test parameter we get still the same problem

# nfs4_setfacl -a A::ldapuser@parisiwin2k3.netapp.com:rwatTnNcCy newkrb5

>> Failed setxattr operation: Invalid argument

we have used our own user and domain name!!!

any ideas ?

thanks a lot

christian

Re: NFSv4 ACLs on RHEL?

parisi — Thu, 08 Aug 2013 15:05:07 GMT

Can you post the output from your commands? And then tail the last 100 lines of /var/log/messages on the client?

Re: NFSv4 ACLs on RHEL?

ccie5863 — Thu, 08 Aug 2013 15:29:16 GMT

Hello Parisi

Please see here our commands and the outputs.

root@pslab-deb1:~# mount

10.99.4.153:/vol/nfsv4 on /mnt/b2 type nfs4 (rw,relatime,vers=4,rsize=65536,wsize=65536,namlen=255,hard,proto=tcp,port=0,timeo=600,retrans=2,sec=sys,clientaddr=10.99.191.41,minorversion=0,local_lock=none,addr=10.99.4.153)

root@pslab-deb1:~#

root@pslab-deb1:~# nfs4_getfacl /mnt/b2

A::root@pslab.nfs:rw

A::OWNER@:rwaDxtTnNcCy

D::OWNER@:

A:g:GROUP@:rwaDxtTnNcCy

D:g:GROUP@:

A::EVERYONE@:rwaDxtTnNcCy

D::EVERYONE@:

root@pslab-deb1:~#

root@pslab-deb1:~# cd /mnt/b2/

root@pslab-deb1:/mnt/b2# nfs4_setfacl -a A::peter@pslab.nfs:rwatTnNcCy /mnt/b2 test --test

Test mode only - the resulting ACL for "/mnt/b2":

A::peter@pslab.nfs:rwatTnNcCy

A::root@pslab.nfs:rw

A::OWNER@:rwaDxtTnNcCy

D::OWNER@:

A:g:GROUP@:rwaDxtTnNcCy

D:g:GROUP@:

A::EVERYONE@:rwaDxtTnNcCy

D::EVERYONE@:

Test mode only - the resulting ACL for "/mnt/b2/test":

A::peter@pslab.nfs:rwatTnNcCy

A::root@pslab.nfs:rw

A::root@pslab.nfs:rwatTnNcCy

A::root@pslab.nfs:rw

A::OWNER@:rwatTnNcCy

D::OWNER@:x

A:g:GROUP@:rtncy

D:g:GROUP@:waxTC

A::EVERYONE@:rtncy

D::EVERYONE@:waxTC

root@pslab-deb1:/mnt/b2# nfs4_setfacl -a A::peter@pslab.nfs:rwatTnNcCy /mnt/b2 test

Failed setxattr operation: Invalid argument

For your Background:

The nfs storage is a netapp FAS – Data Ontap 7.3.7

swsbnap3> options nfs.v4

nfs.v4.acl.enable on

nfs.v4.enable on

nfs.v4.id.allow_numerics on

nfs.v4.id.domain pslab.nfs

nfs.v4.read_delegation off

nfs.v4.write_delegation off

Nfs : domain pslab.nfs

Active Directory : pslab.local

the /var/log/messages file is empty

I can provide a teamviewer session to the onlinesystems

Thanks a lot

Christian

Re: NFSv4 ACLs on RHEL?

parisi — Thu, 08 Aug 2013 15:55:07 GMT

Are you using LDAP on the AD server? Is AD the NFSv4 ID mapping domain?

If it's LDAP, can you run:

# getent passwd peter

Can you restart rpcidmapd service and retry?

Does anything show up in the filer messages?

Can you get a packet trace on the client of the failed ACL set?

Re: NFSv4 ACLs on RHEL?

ccie5863 — Thu, 08 Aug 2013 16:04:04 GMT

Hi Parisi

I am Out oft the Office now and will be back tomorrow

I will Provide all the Infos for you.

Thanks

Re: NFSv4 ACLs on RHEL?

ccie5863 — Fri, 09 Aug 2013 15:53:25 GMT

Hi,

today I changed the Name of my NFS Domain from PSLAB.NFS to PSLAB.LOCAL. So DNS, AD and NFS are using the same Name. Then I installed a Debian 6, because in Debian 7 are some Bugs with rpc.imapd.

It seems Idmapd on Debian is rpc.idmapd. There is no init.d script for the service. I think the only way to start and stop is "start-stop-daemon --start --oknodo --quiet --exec /usr/sbin/rpc.idmapd" and "start-stop-daemon --stop --oknodo --quiet --exec /usr/sbin/rpc.idmapd".

But with "rpc.idmap -f -vvvvv" I can see that it is running now.

For your questions: I have no ldap connection from my debian host. I'm usin krb5. "kinit peter" shows no errors after I entered the Password and with kpasswd i can change the pass of my AD Users.

I can't see anything about my error in the Logfiles. Not on the Debian Host and not on my Filer.

Here are some of my config files and console outputs:

swsbnap3> options nfs.v4

nfs.v4.acl.enable on

nfs.v4.enable on

nfs.v4.id.allow_numerics off

nfs.v4.id.domain PSLAB.LOCAL <---- I changed this today

nfs.v4.read_delegation off

nfs.v4.write_delegation off

swsbnap3> rdfile /etc/exports

/vol/nfsv4 -sec=krb5:sys,rw,root=10.99.191.43

swsbnap3> rdfile /etc/nsswitch.conf

hosts: files dns nis

passwd: files ldap

netgroup: files nis

group: files ldap

shadow: files ldap

swsbnap3> cifs testdc

Using Established configuration

Current Mode of NBT is B Mode

Netbios scope ""

Registered names...

SWSBNAP3 < 0> Broadcast

SWSBNAP3 < 3> Broadcast

SWSBNAP3 <20> Broadcast

PSLAB < 0> Broadcast

Testing all Primary Domain Controllers

found 1 unique addresses

found PDC PSLAB-DC1 at 10.99.191.1

Testing all Domain Controllers

found 1 unique addresses

found DC PSLAB-DC1 at 10.99.191.1

root@pslab-deb3:~# klist

Ticket cache: FILE:/tmp/krb5cc_0

Default principal: peter@PSLAB.LOCAL

Valid starting Expires Service principal

08/09/13 17:14:34 08/09/13 23:54:34 krbtgt/PSLAB.LOCAL@PSLAB.LOCAL

root@pslab-deb3:~# cat /etc/idmapd.conf

[General]

# Verbosity = 0

# Pipefs-Directory = /var/lib/nfs/rpc_pipefs

Domain = PSLAB.LOCAL

root@pslab-deb3:~# cat /etc/nsswitch.conf

passwd: files ldap compat

group: compat

shadow: files ldap compat

hosts: files dns

networks: files

protocols: db files

services: db files

ethers: db files

rpc: db files

netgroup: nis

root@pslab-deb3:~# cat /etc/krb5.conf

[logging]

Default = FILE:/var/log/krb5.log

[libdefaults]

ticket_lifetime = 24000

clock-skew = 300

default_realm = PSLAB.LOCAL

[realms]

PSALB.LOCAL = {

kdc = pslab-dc1.pslab.local:88

admin_server = pslab-dc1.pslab.local:464

default_domain = pslab.local

}

[domain_realm]

.pslab.local = PSLAB.LOCAL

pslab.local = PSLAB.LOCAL

root@pslab-deb3:~# ls -l /mnt/b2

insgesamt 4

drwxr-xr-x 2 4294967294 4294967294 4096 9. Aug 17:39 folder

-rw-r--r-- 1 4294967294 4294967294 0 2. Aug 15:57 test

root@pslab-deb3:~# mount

10.99.4.153:/vol/nfsv4 on /mnt/b2 type nfs4 (rw,addr=10.99.4.153,clientaddr=10.99.191.43)

Thanks for your Help

Re: NFSv4 ACLs on RHEL?

parisi — Fri, 09 Aug 2013 17:04:50 GMT

With NFSv4, simply naming the domain isn't enough. You have to have one to one mapping of username@nfsv4iddomain to UID. The NFS client and NFS server both need to be able to come to the same conclusion about the name.

If you're using no name service server (like LDAP or NIS) and are relying only on local files, then there needs to be an entry for the user on the storage's passwd file and the client's passwd file. These entries must match EXACTLY.

The ID you see in ls -l means NFSv4 isn't even working at all. That resolves to nobody:

http://www.novell.com/support/kb/doc.php?id=7005060

Create an entry for your username in the passwd files on the client and server.

Example:

[root@centos64 /]# cat /etc/passwd | grep peter

peter:x:101:1::/:

[root@centos64 /]# passwd peter

Changing password for user peter.

New password:

BAD PASSWORD: it is based on a dictionary word

Retype new password:

passwd: all authentication tokens updated successfully.

[root@centos64 /]# mount 10.61.72.35:/vol/unix /tmp

[root@centos64 /]# mount | grep /tmp

10.61.72.35:/vol/unix on /tmp type nfs (rw,vers=4,addr=10.61.72.35,clientaddr=10.61.179.150)

[root@centos64 /]# cd /tmp

[root@centos64 tmp]# ls -la

total 12

drwxr-xr-x. 3 root root 4096 Aug 5 12:27 .

dr-xr-xr-x. 26 root root 4096 Aug 8 16:10 ..

drwxrwxrwx. 10 root root 4096 Aug 9 09:00 .snapshot

Note that when I login as "peter" and write a file, it lets me write but the UID is "nobody":

[root@centos64 tmp]# su peter

sh-4.1$ cd /tmp

sh-4.1$ touch file

sh-4.1$ ls -la | grep file

-rw-r--r--. 1 nobody daemon 0 Aug 9 2013 file

Thisis defined in idmapd.conf:

[root@centos64 /]# cat /etc/idmapd.conf

[General]

Domain = domain.win2k8.netapp.com

[Mapping]

Nobody-User = nobody

Nobody-Group = nobody

[Translation]

Method = nsswitch

The fact that the user can't map would also affect me applying NFSv4 ACLs:

[root@centos64 /]# nfs4_setfacl -a A::peter@domain.win2k8.netapp.com:rwatTnNcCy /mnt

Failed setxattr operation: Invalid argument

I have 2 options to fix this.

Option #1: Create an LDAP server to manage your UIDs and users (best option)

Option #2: Add the user to the client's passwd file and server/filer's passwd file (and the entry must be EXACTLY the same):

fas3170-rtp*> wrfile -a /etc/passwd peter::101:2:/:

fas3170-rtp*> rdfile /etc/passwd

root:_J9..LnoxwdFuzh81UF6:0:1::/:

pcuser::65534:65534::/:

nobody::65535:65535::/:

ftp::65533:65533:FTP Anonymous:/home/ftp:

peter:x:101:1:/:

[root@centos64 tmp]# cat /etc/passwd | grep peter

peter:x:101:1::/:

[root@centos64 /]# umount /tmp

[root@centos64 /]# mount 10.61.72.35:/vol/unix /tmp

[root@centos64 /]# cd /tmp

[root@centos64 tmp]# ls -la

total 12

drwxrwxrwx. 3 root nobody 4096 Aug 9 12:20 .

dr-xr-xr-x. 26 root root 4096 Aug 8 16:10 ..

-rw-r--r--. 1 peter daemon 0 Aug 9 12:20 file

drwxrwxrwx. 10 root nobody 4096 Aug 9 09:00 .snapshot

After this, I can apply ACLs, but only if I use the @domain:

[root@centos64 /]# nfs4_setfacl -a A::peter:rwatTnNcCy /tmp

Failed setxattr operation: Invalid argument

[root@centos64 /]# nfs4_setfacl -a A::peter@domain.win2k8.netapp.com:rwatTnNcCy /tmp

[root@centos64 /]# nfs4_getfacl /tmp

A::peter@domain.win2k8.netapp.com:rwatTnNcCy

A::peter@domain.win2k8.netapp.com:rw

A::OWNER@:rwaDxtTnNcCy

D::OWNER@:

A:g:GROUP@:rwaDxtTnNcCy

D:g:GROUP@:

A::EVERYONE@:rwaDxtTnNcCy

D::EVERYONE@:

Once you get NFSv4 working properly and usernames showing up in ls output, ACLs should start working.

Re: NFSv4 ACLs on RHEL?

ccie5863 — Fri, 09 Aug 2013 20:01:36 GMT

Ok, I understand. Then I have to configure a LDAP connection to my AD on the Filer and the Linux Host?

Thanks a lot

Re: NFSv4 ACLs on RHEL?

parisi — Fri, 09 Aug 2013 20:05:30 GMT

No, you need a name mapping service. That could be LDAP, local files or NIS. Just something that the NFSv4 domain can use to map users.

LDAP would be the easiest to set up.

Check out this TR for details on Windows AD LDAP:

http://www.netapp.com/us/system/pdf-reader.aspx?m=tr-4073.pdf&cc=us

It's a cDOT specific TR, but the LDAP portion applies to all environments.