ONTAP Discussions

Kerberos CIFS SVM pre-authentication failure

BenBrazil
11,153 Views

Hi All,

I've enabled AES encryption on one of our CIFS SVM's. 

I ran the vserver cifs security modify -vserver vservername -is-aes-encryption-enabled true

 

This process updates the CIFS server machine account password in Active Directory. The password was updated successfully in Active Directory and I can now see client connections to the CIFS SVM using Kerberos successfully.

 

I have however seen errors in the NetApp event log showing a kerberos pre-authentication failure for the SVM:

 

secd.kerberos.preauth: A Kerberos pre-authentication failure occurred for SVM "vserver" due to invalid credentials for VSERVERNAME$@DOMAIN.LOCAL.

These errors only occur maybe 3 or 4 times in a 24 hour period. The errors only appear on one node out of a 4 node cluster. Additionally, there are no volumes owned by the node reporting the error - apart from one Load Sharing replica volume for the SVM root.

User CIFS connections are not impacted, so the only indication of the issue are the event log errors.

The error message states the following as the reason why there is an authentication issue@

 

This message occurs when invalid credentials are provided for an Active Directory user or the machine account password is out of sync with the credentials set in the Active Directory.

 

The resolution is to run the password reset command to update the password again.

 

I have run vserver cifs password-reset -vserver vservername which did update the Active Directory password again, but I still have the same error message.

 

NetApp support suggested to run an alternative cifs domain password reset -vserver vservername command. This hasn't fixed the issue either.

 

Any suggestions as to what to do next? The cluster is running at 9.12.1P2.

 

Thanks

Ben

 

 

 

12 REPLIES 12

Ontapforrum
11,106 Views

Found some related kbs: (If you have already seen these kbs, then it may be worth just raising a ticket with NetApp to investigate this issue)
https://kb.netapp.com/onprem/ontap/da/NAS/secd.kerberos.preauth_event_is_being_logged
https://kb.netapp.com/onprem/ontap/da/NAS/Kerberos_error_secd.kerberos.preauth_due_to_out-of-sync_machine_account_password_due_to_invalid_spn

 

Are all users logged in via Kerberos (not using ntlm):
::> vserver cifs session show -fields auth-mechanism

BenBrazil
11,089 Views

Thanks for the reply.

All new user sessions since enabling AES encryption are using Kerberos. Previously they were using NTLMv2. The errors only appear a few times a day and don't correspond to when users connect a new CIFS session. I would expect to see hundreds all of the time if the error log events were as a result of each CIFS session.

 

I've got an open case with support, but they've just said to run the cifs domain password reset -vserver vservername again because they think I'm using the wrong password

 

"So kindly run the command again and make sure you are using the right credentials and that will stop the alerts."

 

That resolution doesn't make sense to me as when running that command, you are only prompted to enter an account with permission in Active Directory to make password changes. There is no prompt to enter the password for the CIFS server object in AD. The password does get changed each time in AD when running the command, so I at least know it is doing what it is meant to be doing in AD.

 

I've not found any Kerberos or SPN related errors in the secd log. I did check the SPN with Support and they didnt think there was an issue there either.

 

Thanks,

Ontapforrum
11,035 Views

Yes, that's a correct observation. : "That resolution doesn't make sense to me as when running that command, you are only prompted to enter an account with permission in Active Directory to make password changes." 

 

Actually, you don't change the CIFS Server machine-account password , it is refreshed automatically based on the AD policy. You just need to "reset" the password if it has been manually changed on the AD side without NetApp(SVM) knowledge. In this situation, you just have to reset it by running the command as suggested. The Password you provide while running this command, is only needed to authenticate to AD to make sure you have the permission to do so.

 

Following command resets the password for the "CIFS server" on the supplied vserver:
::> vserver cifs domain password reset -vserver <server>

 

Basically, once that command is run,  you don't need to do anything else on the password side. Could you tell us if there has been 'fresh' generated events in the last 24 hrs ?

BenBrazil
10,879 Views

Hi,

 

There have been some fresh generated events over the past few days, none for the past couple of days however.

I've taken a tcp trace and NetApp support are looking through to try and see what is happening.

lplataforma-sis
9,471 Views

Any news abouyt this? it is happening also to me, version 9.9.1P15.

 

'reseating' the password didn't help:

::> vserver cifs domain password reset -vserver <server>

::>  vserver cifs password-reset -vserver <server>

 

I'm still getting these errors:

 

secd.kerberos.preauth: Kerberos pre-authentication failure due to out-of-sync machine account password for vserver

AIRBORNERIC1989
8,854 Views

Any update?  Running into the same thing 9.12.1P5

HdJ
8,790 Views

We have the same issue on 9.12.1P4, tried everything but no solution yet. Manual password updates work but does not resolve the issues.

MaGr
8,573 Views

Hi,

we had a similar issue with authentication that passes the domain tunnel.

 

It's fixed with 9.11.P12. Bug ID 1513324

Marcus

bauernfeind
7,960 Views

Hi,

 

we have a similar issue, in connection with kerberized NFS.
Sometimes, when a user tries to access a kerberized LIF, the error message is triggered.
on the KDC, it is first shown that the SVM tries to authenticate via kerberos, which fails. Afterwards it falls back to NTLM and succeeds.

We are on 9.12.1P10 (but i first encountered the problem end of september which 9.12.1P4)

I already opened a still open case in November, but not much has come out of it.

darkavich
7,896 Views

I have the same problem that has got worse since we upgrade to 9.12.1P9 from 9.11.1. Support is telling us that it's a domain trust issue but our domain admins claim that's not true. Support is trying to push me down the path of turning off domain discover, but that's not where the problem lies nor makes any sense to me. Hopefully we can get to the bottom of this soon.

bauernfeind
5,018 Views

Hi, I just want to reply that our error is fixed after enabling ldap session sealing for the ldap client.

ldap client modify -client-config <LDAP-PROFILE> -session-security seal

https://mysupport.netapp.com/site/bugs-online/product/ONTAP/BURT/1522219

toaster
26 Views

Using seal (or sign) for LDAP client session security partially fixed the problem for us.  However, we were still seeing sporadic Kerberos preauth failures (due to invalid SVM credentials) in the event logs at random times, after applying that setting.  We gathered detailed secd traces and NetApp opened the following bug, related to concurrent I/O to a local Kerberos cache file by competing RPCs.  It is suspected that a race condition between those parallel RPCs is causing the preauth errors in our particular situation.  There is no fix yet.

https://mysupport.netapp.com/site/bugs-online/product/ONTAP/JiraNgage/CONTAP-358837

Public