topic Re: Total latency for FCP LUN in Network and Storage Protocols

Total latency for FCP LUN

SupraJari — Wed, 04 Jun 2025 12:10:55 GMT

Hello, I'm trying to figure out how to get total latency for a FCP LUN. I have following numbers from perfstat:

fc_lif_1: avg_read_latency:1677.41us
fc_lif_1:avg_write_latency:1906.61us
fc_lif_1:avg_other_latency:261.62us

Taking a look at particular LUN-numbers:

lunx:avg_read_latency:1.78ms
lunx:avg_write_latency:2.42ms
lunx:avg_unmap_latency:0ms
lunx:avg_other_latency:0.92ms

Combining FCP lif read latency 1.68ms + LUN read latency 1.78ms, would it be right to say complete FCP latency "inside" the controller is 3.46ms? What I'm trying to get is total time spent when FCP read request arrive at controller and leaves FCP port to user.

EDIT: Does LUN or FCP latency include compression, dedup etc?

Cheers,

Sup

Re: Total latency for FCP LUN

GidonMarcus — Fri, 25 Oct 2019 16:00:36 GMT

the LUN latency already include the network "round trip" that i believe you refer to - see:

LUN Latency ~= VolumeLatency * ROUNDUP(OperationSize / 64KB) + NetworkRoundTripTime * (ROUNDUP(OperationSize / SegmentLength) - 1^*)

So you don't need to add the LIF latency yourself.

https://kb.netapp.com/app/answers/answer_view/a_id/1002524/loc/en_US#__highlight

based on it. i also assume that it include the latency Dedup and other volume/aggr level efficiencies add/reduce.

Re: Total latency for FCP LUN

SupraJari — Mon, 04 Nov 2019 08:33:12 GMT

Hi Gidi,

and thanks for your valuable answer.

Taking a look at the link you posted, it has intresting section:

=====

Initial test with 64KB read operations (host measured latency = 103ms, controller lun latency == volume latency ~ 0.06ms):

fas01*> stats show -r -n 1 -i 5 volume:demo2:read_latency volume:demo2:read_ops lun:/vol/demo2/lun2-BLH0G?BQgK/T:avg_read_latency lun:/vol/demo2/lun2-BLH0G?BQgK/T:read_ops volume:demo2:read_latency:61.42us volume:demo2:read_ops:9/s lun:/vol/demo2/lun2-BLH0G?BQgK/T:avg_read_latency:0.04ms lun:/vol/demo2/lun2-BLH0G?BQgK/T:read_ops:9/s

Test with 128KB read operations (host measured latency = 205ms, controller LUN latency of 101ms includes 1 network round trip):

fas01*> stats show -r -n 1 -i 5 volume:demo2:read_latency volume:demo2:read_ops lun:/vol/demo2/lun2-BLH0G?BQgK/T:avg_read_latency lun:/vol/demo2/lun2-BLH0G?BQgK/T:read_ops volume:demo2:read_latency:44.71us volume:demo2:read_ops:9/s lun:/vol/demo2/lun2-BLH0G?BQgK/T:avg_read_latency:101.58ms lun:/vol/demo2/lun2-BLH0G?BQgK/T:read_ops:4/s

Test with 256KB read operations (host measured latency = 408ms, controller LUN latency of 302ms includes 3 network round trips):

fas01*> stats show -r -n 1 -i 5 volume:demo2:read_latency volume:demo2:read_ops lun:/vol/demo2/lun2-BLH0G?BQgK/T:avg_read_latency lun:/vol/demo2/lun2-BLH0G?BQgK/T:read_ops volume:demo2:read_latency:47.02ussss volume:demo2:read_ops:10/s lun:/vol/demo2/lun2-BLH0G?BQgK/T:avg_read_latency:302.62ms lun:/vol/demo2/lun2-BLH0G?BQgK/T:read_ops:2/s

=====

First testing is OK, but I don't understand how one round trip (which I understand is host requesting two read operations to get two 64k chunks), can add 100ms to the lun latency. And to read 256 (4*64k) adds 2 * 100ms. Must be testing thing or something, those latencies are grazy..

Re: Total latency for FCP LUN

GidonMarcus — Mon, 04 Nov 2019 15:37:27 GMT

indeed - the latency added to prove the point. they mention:

In the example below, a Linux iSCSI client is used with network latency artificially added by running the following command:

# tc qdisc add dev eth0 root netem delay 100ms