Powershell 'Invoke-NcSsh' cmdlet triggers 'invalid login' EMS event yet completes successfully.

[BenCoughtry]

Hello friends, hope you're all safe and well!

I'm having a problem where automation scripts using NetApp Powershell Toolkit are running successfully, but upon first cluster login triggers an 'invalid ssh login' event confirmed in ontap logs, however all commands complete successfully returning desired data.  Functionally, the login error appears to be a false positive because the command completes.

History and t-shooting steps:

This started happening on multiple clusters after I generated new self signed SSL certs, as previous certs had expired.  Confirmed credentials used are correct and all previous caches were cleared.  I also cleared the certificate cache used by putty but no behavior change.  Same behavior occurs sourcing Invoke-NcSsh from a completely different server which was never used for netapp automation;  I installed the latest putty msi and latest netapp powershell toolkit, and ran Invoke-NcSsh command (output below) which completed successfully, yet still triggered an 'invalid ssh login' event.

Powershell copy & paste:

PS C:\Windows\system32> Import-Module DataONTAP
PS C:\Windows\system32> $creds = Get-Credential

cmdlet Get-Credential at command pipeline position 1
Supply values for the following parameters:
Credential
PS C:\Windows\system32> Invoke-NcSsh -ControllerName [xx ommitted xx] -Command "version" -Credential $creds

NcController : [xx ommitted xx]
Value :

Last login time: 9/30/2021 17:22:51

Unsuccessful login attempts since last login: 1

NetApp Release 9.6P5: Fri Dec 13 18:21:56 UTC 2019

OnTap logs:

Message: security.invalid.login: Failed to authenticate login attempt to Vserver: [xx ommitted xx], username: admin, application: ssh.

Notice the 'Unsuccessful login attempts since last login' in the powershell output above.

Details:

-User is 'admin'.

-Using putty binary file to SSH into the cluster does NOT trigger an invalid login alarm - the alarm only occurs with powershell 'Invoke-NcSsh'.

-I followed documented steps for deleting and regenerating self signed SSL certs and assign them to the cluster vserver, but this problem has been happening ever since I did this.

Has anyone run into this before?  Any ideas would be appreciated.  Thanks so much!

-Ben

[hmoubara]

Hello,

Can you verify which host is trying to access and getting the login denied. You can check the ems logs for specific event:

cluster1::*> event log show -event *invalid.login*

cluster1::*> event log show -event *loginDenied*

Thanks 

[BenCoughtry]

Thanks for the reply.  Any powershell source results in this error for affected clusters.  Interestingly, 'connect-NcController' and other powershell cmdlets does NOT trigger this alarm, only 'Invoke-NcSsh'.

I have tried multiple versions of powershell module and putty, from servers that never access the clusters before, and still behavior exists.

This does NOT happen on newer clusters I did not regenerate certs on, so I'm wondering if there is some table somewhere in OnTap that has to be cleared?

[ttran]

Hello @BenCoughtry ,

Just to rule out the basic configuration didn't get altered somehow, can you verify the Application: SSH is listed for the specific username and vserver in the following output from the Cluster Shell: " security login show".

This KB also details how to update and check if the certificates are updated properly on the entire cluster and SVMs. This is to rule out the possibility that one or some of the SVMs didn't update properly and giving you mixed results with 'Invoke-NcSsh'.

How to renew a Self-Signed SSL certificate in ONTAP 9 

Regards,

Team NetApp

[BenCoughtry]

Thanks for the reply!  Yes, verified with netapp support that the new certs were generated properly.  Replaced again during a call with netapp support but no behavior change.  Events lots below show SSH error with user 'admin':

san12c-cluster::*> event show -severity *
Time Node Severity Event
------------------- ---------------- ------------- ---------------------------
10/11/2021 20:45:48 san12c NOTICE sshd.auth.loginDenied: message="Failed keyboard-i nteractive / pam for admin from 10.240.54.20 port 54393 ssh2 "
10/11/2021 20:45:44 san12c ALERT security.invalid.login: Failed to authenticate lo gin attempt to Vserver: san12c-cluster, username: admin, application: ssh.

[gmontury]

Hello everyone,

I meet the same issue.

This is my context:

cdot-k3::> version
NetApp Release 9.9.1P11: Tue Aug 02 19:12:28 UTC 2022

PS D:\Sources\NetApp.ONTAP> Get-Module -Name DataONTAP
ModuleType Version Name ExportedCommands
---------- ------- ---- ----------------
Manifest 9.0.8.... DataONTAP {Add-NaAggr, Add-NaCifsShare, Add-NaCredential, Add-NaFpolicyExtension...}

Value :

Last login time: 4/19/2023 11:30:32

Unsuccessful login attempts since last login: 2

I have update the ONTAP module in version 9.12.

PS D:\Sources\NetApp.ONTAP> Get-Module -Name NetApp.ONTAP

ModuleType Version Name ExportedCommands
---------- ------- ---- ----------------
Manifest 9.12.1.... NetApp.ONTAP {Add-NaAggr, Add-NaCifsShare, Add-NaCredential, Add-NaFpolicyExtension...}

Now the issue is fixed:
Last login time: 4/19/2023 14:58:48

I hope that it could be help you.

Regards,

Gerald

[BenCo]

Thanks for your reply!  This has indeed fixed the issue for myself as well. To note, in order to upgrade my toolkit I had to also enforce TLS minimum version 1.2 on my PowerShell session, but the 'invalid login' message has disappeared.