2012-08-01 11:37 AM
I cannot find much information about this check and wasn't sure if anyone can provide some more insight into these nfs read latency checks and how useful they actually are or if anyone is actually using them.
Any information would be greatly appreciated.
Solved! SEE THE SOLUTION
2012-08-01 11:42 AM
That event you're seeing is generated by Performance Advisor. It is a tool that comes with OnCommand 5.x Unified Manager (previously known as DFM or Operations Manager). Someone has previously created a performance threshold on volume NFS read latency. When that performance threshold is breached for the specified interval, it generates the event. Once NFS read latency goes back down below the threshold, it will automatically clear the event.
This information is very useful if you're concerned with that volume's NFS read latency. If you're not concerned about it, you can delete the threshold and it will stop generating the events.
2012-08-01 11:44 AM
Yea that makes sense, my concern is the threshold is too low because they are generating a lot and I know our latency isn't that bad, do you know of any places that have good recommendations for NFS latency by chance so I can maybe adjust them if they are too low on the threshold?
2012-08-01 01:41 PM
NFS latency is completely subjective. Each customer & each NFS application will have its own acceptable level of latency. 10 ms may be fine for one NFS environment and completely unacceptable for another. That's why you'll never see a "good general recommendation" on NFS latency thresholds.
The correct person to ask is the end-consumer of your NFS storage. They should be telling you when latency becomes unacceptable because they'll feel it in their application (Oracle DB, CAD application, etc..). If they're not complaining when latency exceeds your current threshold, then I'd recommend you increase the threshold.
2012-08-02 12:25 PM
Some apps (for instance Oracle DB) have tunables on what to expect for their storage latency. Please keep in mind that if you're running VMs on top of NFS you need to take the end-to-end latency into account, not just what's there from NFS. In that case you probably want to measure from inside the VM.
Generally latency of less than 4ms I'd considered pretty fast but YMMV. On a filer there can be a wide difference because 1) It's writing to a file system (and NFS requires more steps to be done in software than on a traditional block based storage device) so if your filer's CPU is busy it can impact latency of IO more than dumb storage protocols, 2) NVRAM can accept writes very quickly...until it fills up. If you have constant back to back checkpointing your right latency can skyrocket...but normally writes are *very* low latency on a properly sized filer and finally 3) Reads can be cached. Especially with FlashCache/PAM cards you can end up with super low latency most of the time (serving the data out of the cache) and then weird spikes when you have a cache miss. Of course the cached data saves you from seeking for it on the spindles so you end up getting faster reads even for the stuff that comes from disk.