High read latency on filer

[daniel_kaiser]

Hi,

I'm noticing  high read latency numbers for NFS on my filer via System Manager 1.1.  I'm fairly new to the NetApp world, so i really don't know what to check to figure out what's causing the high latency.  Any help would greatly be appreciated.

We have a clustered 3140 with 5 shelves of 450GB FC 15k drives and a shelf of 1TB SATA drives.  One controller has a single aggregate of FC drives (33 disks) while the other controller has two aggregates, one with all SATA disks and the other with the remaining FC disks.  This high read latency appears on both controllers

We are using this SAN primarily for virtualization (vSphere).  Both protocols are currently being used, NFS and FC.  The green graphs in the picture is for NFS.

[daniel_kaiser]

I found this article, http://communities.netapp.com/groups/chris-kranz-hardware-pro/blog/tags/latency, and used the stats command to show the below.

na1_ds_nfs1 is a vmware datastore hosting roughly 20 VM machines.

netapp1*> stats show -n 50 -i 1 volume:na1_ds_nfs1:read_latency
Instance read_latency
                   us
na1_ds_nfs1     11647.00
na1_ds_nfs1            0
na1_ds_nfs1      7669.33
na1_ds_nfs1            0
na1_ds_nfs1            0
na1_ds_nfs1            0
na1_ds_nfs1            0
na1_ds_nfs1            0
na1_ds_nfs1            0
na1_ds_nfs1            0
na1_ds_nfs1        23.00
na1_ds_nfs1            0
na1_ds_nfs1            0
na1_ds_nfs1            0
na1_ds_nfs1            0
na1_ds_nfs1            0
na1_ds_nfs1            0
na1_ds_nfs1            0
na1_ds_nfs1            0
na1_ds_nfs1            0
na1_ds_nfs1            0
na1_ds_nfs1            0
na1_ds_nfs1     13915.00
Instance read_latency
                   us
na1_ds_nfs1      4741.00
na1_ds_nfs1            0
na1_ds_nfs1            0
na1_ds_nfs1            0
na1_ds_nfs1            0
na1_ds_nfs1            0
na1_ds_nfs1            0
na1_ds_nfs1            0
na1_ds_nfs1            0
na1_ds_nfs1            0
na1_ds_nfs1            0
na1_ds_nfs1            0
na1_ds_nfs1            0
na1_ds_nfs1            0
na1_ds_nfs1            0
na1_ds_nfs1            0
na1_ds_nfs1            0
na1_ds_nfs1        14.00
na1_ds_nfs1            0
na1_ds_nfs1            0
na1_ds_nfs1            0
na1_ds_nfs1            0
na1_ds_nfs1            0
Instance read_latency
                   us
na1_ds_nfs1            0
na1_ds_nfs1            0
na1_ds_nfs1            0
na1_ds_nfs1     12255.00

[jjbeckham · Accepted solution]

That's actually really good latency. 12 milliseconds isn't bad at all for NFS. NetApp is displaying the graph you posted in microseconds.