Subscribe

CIFS Ghost-Like Performance Issue

Good Morning All,

I am running into a weird ghost-like issue with some users accessing a certain CIFS share.  I have a pair of FAS3240s running ONTAP 8.0.2P6.  The users use a CAD like program, Microstation, which calls several smaller files from the CIFS share where the project is located.  Most of the time it runs fine and there are no issues, but some days the users see between a 5-15 second lag for actions that are normally instant, there is no real pattern to when it works vs lags.  This happens with multiple users at the same time, and doesn't seem to matter who the user is or what permissions they have.  If the application files are copied locally then it works fine.  I have engaged NetApp support and all they found was that I have a close to full volume (not the one with the CIFS Share), and I have Flow Control enabled ( Plan to turn it off, just haven't yet).  I ran WireShark on one of the user's PCs during the lag, and during normal operation.  During the lag I see several of these messages.

SMB93Trans2 Response, QUERY_PATH_INFO, Error: STATUS_OBJECT_NAME_NOT_FOUND

and

SMB93Trans2 Response, QUERY_PATH_INFO, Error: STATUS_OBJECT_PATH_NOT_FOUND

Some research has turned up that this error is relative to SMB/SAMBA performance issues, but none of my research points it back to NetApp issues.  Has anyone experienced any issues like this or could you point me in the right direction??

Thank you,

John

Re: CIFS Ghost-Like Performance Issue

Is this link helpful: http://support.microsoft.com/kb/2628582

Just a shot in the dark

Re: CIFS Ghost-Like Performance Issue

Your problem sounds EXACTLY like a problem that we've run into at two of our facilities and we have found the solution for our situation. The difficulty was the random nature of the problem, but when it occurred it had significant impact on our users' ability to access their file systems.  We took a pktt trace and saw similar errors; SMB2 135 Create Response, Error: STATUS_OBJECT_NAME_NOT_FOUND or SMB2 135 Ioctl Response, Error: STATUS_INVALID_DEVICE_REQUEST.  The big clues for us were the TCP 70 [TCP Dup ACK #####] 445 errors, the TCP 1518 [TCP Out-Of-Order] errors, and the [TCP ACKed unseen segment] errors. 

 

We found that there is an incompatibility between NetApp systems using Fastpath (enabled by default) and Nexus switches using virtual port channeling (vPC) without the vPC peer-gateway feature enabled. The vPC peer-gateway was introduced in NX-OS 4.2.1 for the 7000 series switches and 5.0(3)N1(1) for the 5500 series switches after the problem was discovered.  If you are using NetApp attached to Cisco Nexus 5500 or 7000 series switches, be sure you either enable the peer-gateway on your port channels, or disable Fastpath on your NetApp system.  The following links provide additonal details.

 

I hope this is helpful for others.

Mike W.

 

KB# 2013848 Version: 6.0 Published date: 03/27/2014

Related Links:

  • KB 1012623: OEM: How to disable ip.fastpath in Clustered Data ONTAP
  • KB 3011178: How does a multimode vif interact with ip.fastpath?