WFA job failure

[Madhukar]

We have a custom workflow that sanitize all the disk in a 7 Mode system. The workflow had been working sucessfully for more than 2 year from now. However it failed with below error now.

An established connection was aborted by the software in your host machine.

The workflow first sanitize all the disk on Node A and then starts with Node B. It worked for  Node A without any issues and now it failed with Node B

I have attached the logs for the same.

Could you please help to resolve this issue.

[abhit]

Can you please call up NetApp Support and submit a call rec.

Regards

Abhi

[sinhaa]

@Madhukar

Kindly provide details:

1. Your WFA version where the failure is seen now?

1.1. Did you upgrade your WFA version? Is the failure seen now after upgrading?

2. Your 7Mode ONTAP version

3. Have you upgraded your ONTAP recently? Or you modified any security settings/options?

4. The error indicates thatt your custom command is trying to do Ssh to the cluster using Invoke-NaSsh. Confirm on this?

5. Provide your custom workflow exported in .dar format.

sinhaa

[Madhukar]

Hi Sinhaa,

Thank you for looking into this.

Please find my answers below.

1. Your WFA version where the failure is seen now?

      3.1.0.0.2P2

1.1. Did you upgrade your WFA version? Is the failure seen now after upgrading?

        No I didn't upgrade the WFA version. This workflow was working fine for more than 2 years from now. 

2. Your 7Mode ONTAP version

      NetApp Release 8.2.5 7-Mode: Wed Jul 19 03:55:53 PDT 2017

3. Have you upgraded your ONTAP recently? Or you modified any security settings/options?

      This workflow works only for 7 Mode, Hence I have removed cDOT and installed 7 Mode on this storage system.  We have installed a fresh 7 Mode ONTAP and no security settings/options changed.

4. The error indicates thatt your custom command is trying to do Ssh to the cluster using Invoke-NaSsh. Confirm on this?

       Yes

5. Provide your custom workflow exported in .dar format.

[sinhaa]

@Madhukar

Okay I know the problem and I also know how to resolve it. Invoke-NcSsh is a highly popular cmdlet and hence its failure is affecting many users.

I'll post it in some time. I request some patience.

sinhaa

[sinhaa · Accepted solution]

@Madhukar

Cause of the failure: I'm not absolutely certain here, but from what I've seen its appears that the newer versions of ONTAP with tightened security or when high security is enabled on the ONTAP, Invoke-NcSsh failes to connect. Invoke-NcSsh is a SSH client that uses 3rd party assemblies to establish ssh connections which don't work with this kind of secure ONTAP and hence the error.

The solution:

I don't know what or when (if at all  because of the 3rd party dependency ) DataONTAP PSTK can fix this problem, but your commands can work as before if you deploy the below fix.

The Solution Logic: Some of you might have observed that though Invoke-NcSsh fails, you can still connect to the ONTAP via SSH using other ssh clients like openssh or PuTTy.  So now, if I can get my Invoke-NcSsh to use PuTTY to connect to the ONTAP instead of its own failed assemblies, and get the same response object in return, my problem is solved. I've used PuTTY's plink.exe to make it work for me.

Do the following:

1. Install PuTTy from https://www.putty.org/ . Ive tried with version 0.64, 0.69, 0.70 and it works for me. You can take any latest version as you wish.

2. Download the attached file Invoke-NcSsh.txt and copy it to  your \WFA\PoSH\Modules folder

3. Open this file in any Editor and Find the line :

$env:Path += '; C:\Program Files (x86)\PuTTY\'

Modify this path according to the installation path of your Putty. The above location should contain plink.exe . If you don't give it correctly you will get error plink.exe not found.

4. Save the file with changing the file extension to .psm1 i.e. the file should now be a powershell module Invoke-NcSsh.psm1

5. That's all, Done.

All your commands will continue working as before. The new definition of Invoke-NcSsh overrides that of the DataONTAP PSTK. It also returns the response object exactly same as done by the old one. Your commands will now work seamlessly.

Have fun!.

sinhaa