PowerShell Connection Timeout

[alborz]

Hi,

System was reporting PowerShell Connection Timeout

Connection timeout at the Get-NcVol command:
get-ncvol : Connection to FAS3220CB-m on port 443 for protocol HTTPS timed out.
+ get-ncvol
+ ~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (FAS3220CB-m:NcController) [Get-NcVol], NaConnectionTimeoutException
    + FullyQualifiedErrorId : ApiException,DataONTAP.C.PowerShell.SDK.Cmdlets.Volume.GetNcVol

Increasing the $global:CurrentNcController.TimeoutMsec does not have any effect

We provided the following action plan;

Open an RDP session to the WFA server.
Go to <InstallDirWFA>\netapp\WFA\PoSH\Modules\WFAWrapper
Backup WFAWrapper.psm1 file to a separate location.
Edit the original WFAWrapper.psm1 file.
Change the timeout setting from [int]$Timeout=60000 to [int]$Timeout=300000 under the following three categories/functions inside the file:
Connect-WfaCluster, Connect-WfaController, and Connect-Controller.
Save the file.

another error after doing the steps;

In about 50% of script runs, I get this error on the Get-NcVol command:
get-ncvol : API invoke failed.
+ get-ncvol
+ ~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (FAS3220CB-m:NcController) [Get-NcVol], NaException
    + FullyQualifiedErrorId : ApiException,DataONTAP.C.PowerShell.SDK.Cmdlets.Volume.GetNcVol

The Connect-NcController command always works without troubles.
On three other filer, the script is also running without errors all the time.

Question:

why changing timeout settings on the WFA server had any effect. Do you have an explanation on this? Or is it possible only through the adaptation in the script?

Thanks

[mbeattie]

Hi,

Have you try explicitly increasing the timeout within the WFA credentials for the clusters that are erroring? In WFA click on the Execution tab, Click on Credentials in the left navigation menu then select the cluster and click "override defaults" and update the timeout values. EG

This isn't a definative answer, just a troubleshooting suggestion. There are a number of issues that might be causing your timeout, network latency, routes, firewalls etc.

It's difficult to say without knowledge of your environment. Do the systems that timeout have a high volume count? Are these systems under a high workload?

Are you able to replicate the issue (externally to WFA) in a powershell script?

/Matt

[mbeattie]

Just as an example, are you able to reproduce the issue from your WFA server using the following script\commands (changing the variables to meet your requirements)

PS C:\> cd 'C:\Program Files\NetApp\WFA\PoSH'
PS C:\Program Files\NetApp\WFA\PoSH> .\profile.ps1
PS C:\Program Files\NetApp\WFA\PoSH> $credentials = Get-Credential -Credential admin
PS C:\Program Files\NetApp\WFA\PoSH> $vservername = "vserver1"
PS C:\Program Files\NetApp\WFA\PoSH> Connect-NcController -Name cluster1.testlab.local -HTTPS -Credential $credentials

Name                 Address           Vserver              Version
----                 -------           -------              -------
cluster1.testlab.... 192.168.100.2                          NetApp Release 9.1: Thu Dec 22 23:05:58 UTC 2016

PS C:\Program Files\NetApp\WFA\PoSH> Get-NcVol -Vserver $vservername

Name                      State       TotalSize  Used  Available Dedupe Aggregate                 Vserver
----                      -----       ---------  ----  --------- ------ ---------                 -------
cifs_data_001             online       100.0 GB    5%    95.0 GB False  testc1n1_aggr1            vserver1
vserver1_root             online         1.0 GB    5%   971.4 MB False  testc1n1_aggr1            vserver1

If it fails connecting to the cluster add the "-Timeout" parameter to the "Connect-NcController" and keep incrementing it too see if that's the issue

/Matt

[alborz]

Hi Matt,

Thanks for reply.

The point is, we do not use WFA in this system and any changes to it should not have effect on PowerShell Connection Timeout. This is the question; why changing WFA has solved the PowerShell Connection timeout?

Thanks

[manuel_s]

Hello!

I am fimiliar with this case.
Changes to the WFA obviously had no effect. The script just worked a few times after the WFA changes by accident . Afterwards, however, there were very often the connection timeouts again.
The idea, that it has something to do with the WFA was basically a misunderstanding. In fact, it has always been a simple PowerShell script, which is started by any client or host.

Unfortunately, this problem still exists. Are there any solution ideas that could be followed?

Thanks and regards,

Manuel

[mbeattie]

Hi Manuel,

It was mentioned previously "On three other filers, the script is also running without errors all the time". A few thoughts for troubleshooting...

• What's differrent about the controllers your script does work on compared to the one you recieve timeout errors?
• Are they all running the same version of ONTAP?
• Are they in the same physical data center, are they on same network?
• Is there any load balancers or firewalls between them?
• What version of the PowerShell Toolkit are you running (See Get-NaToolkitVersion).
• Do the controllers run the same workload?
• Have you tried testing a connection using the manageontap.dll from the NMSDK instead of PowerShell and if so do you still recieve a timeout?
• Is the script always invoked from the same client?

It's difficult to determine what might be causing the problem without knowing your environment or looking at what your script is attempting to do so...

• Are you able to provide your script and an explanation of it's intended purpose?
• Does it always error at the same point on the same controller?
• Does your script containing verbose logging to help debug the error?

/Matt

[manuel_s]

Hi Matt!

Thanks for the many suggestions for troubleshooting.

Since for this system an update from 8.3.2 to 9.1 had been planned anyway, we have now finally managed to upgrade the system.
It seems, that the script run smoothly for now.

I guess, it was due to the high utilization of the system, which has now declined slightly due to the update.

Regards,

Manuel