we have a problem with one of our filers. It is a two node cluster with 8060 nodes.
We are getting from time to time theser errors:
Event: Nblade.nfsV4PoolExhaust: NFS Store Pool for Client exhausted. Associated object type is CLUSTER_NODE with UUID: qqqqqqq.
Message Name: Nblade.nfsV4PoolExhaust
Sequence Number: 12361461
Description: This message occurs when one of the NFSv4 store pools is exhausted.
Action: If the NFS server is unresponsive for more than 10 minutes after this error occurs, contact NetApp technical support.
It seems, that this error doesn't cause any issues, but I would like to get rid of it. You never know, when it would cause an outage.
As I understand this error correctly, it looks like, that there are no client connections left on that node. Is that correct?
Yes, we have several hundreds of VMs, using several different volumes on this node. On the other node, there are lesser client connections...
Yes, NFSv4 clients will create multiple/many connections to a node based on how many files they are accessing. Do any of those clients that you listed have an overwhelmingly large amount of NFSv4 locks compared to the other clients? The KB has some additional information about how to view the Lease Count (objects owned for a single client) for NFSv4 clients).
Answering that question will help to concentrate the troubleshooting efforts.
We had similar issue on our NetApp C-Dot systems with NFS v4.0 but after moving on NFS v4.1. it did not appear from last week. We could not even find a clear indication or any statement from NetApp Technical support that this is a fix however after moving to NFS v4.1 this error is gone.