Message: secd.dc.out.of.pipe.instances

[NETAPPNewby]

Has anyone seen this error?

We are getting these from mulitple clusters at different locations.

All errors point to the same DC.  Spoke with AD team and they noticed hight CPU during the day and also saw a Access Denied error associated with the account (CIFSservername$), the name with a $ sign at end. 

We have started getting this error: Message: secd.dc.out.of.pipe.instances: MSRPC for Vserver name to DC name:\netlogon was retried 968 times (delay: 1081793 usecs).

Description: This message occurs when the Windows Domain Controller (DC), while responding to an MSRPC request from Data ONTAP(R), returns an error indicating that it ran out of pipe instances in the listening state. This can happen when the requests from Data ONTAP are received by the DC during a timing window when the limited number of listening pipe instance are already allocated to other requests. Such errors received by Data ONTAP might result in longer CIFS/NFS authentication and/or access times, potentially resulting in CIFS/NFS client timeouts.

[ManpreetS · Accepted solution]

This error is seeing because of limitation of open pipes on DC. In this error message its netlogin pipe which is getting refused from DC.

This is happening because DC ran out of pipe instances in the listening state for that specific time.

Please check with MS support for possible workarounds.

One workaround is to add more DC but not sure if newly added DCs will be able to handle all calls.

FYI:  NetApp artcle describes about the error

[S-Sahu]

Hi @ManpreetS ,

One of our valued customer is having the same issue, we have raised a case with Microsoft and they are not able to see any error events on the DCs, this error is generated by one of the CIFS SVM, also does it register these errors on the NetApp side and which log should I look into? 

MS is saying we need to go back to NetApp,

Any help?

Thanks

Sukant

[ManpreetS]

Event Description:
This message occurs when the Windows Domain Controller (DC), while responding to an MSRPC request from Data ONTAP, returns an error indicating that it ran out of
pipe instances in the listening state. This can occur when the requests from Data ONTAP are received by the DC during a timing window, when the limited number of
listening pipe instance are already allocated to other requests. Such errors received by Data ONTAP  might result in longer CIFS/NFS authentication and/or access times, potentially resulting in CIFS/NFS client timeouts. The event will also include the SVM that encountered the failure, the domain controller (DC), pipe on the DC, and the number of retries.

I am not sure if MS support had checked for listening instances at the time of issue from DC end. However, the best to understand this or confirm this is to collect simultaneous packet traces from storage and DC end at the time when error is being reporting. You can run rolling traces from both end.

For further troubleshooting or analyzing the traces, please open a NetApp support case.

[ManpreetS]

Even though this doesn't have much info but have relevant details from NetApp perspective:

https://kb.netapp.com/Advice_and_Troubleshooting/Data_Storage_Software/ONTAP_OS/EMS_message_alert_%22secd.dc.out.of.pipe.instances%22