what is the difference between iscsi lun and FCP lun

nikhilm · ‎2008-04-01

triantos · ‎2008-04-01

A LUN is a LUN is a LUN. Structurally, there are no differences between the two. The difference lays in the protocol which is used to access them. You can umap an FCP LUN and map it to an iSCSI initiator or vice versa.

In fact, with the ONTAP DSM 3.2 you can have a mix of iSCSI and FCP paths to the same LUN on a windows box.

nikhilm · ‎2008-04-01

There must be an performance difference between them. like time taken to access iscsi lun is greater than fc lun.

triantos · ‎2008-04-02

The performance of an iSCSI LUN vs an FC LUN is dependent upon many things, the however, the structure of the LUN itself is not one of them. For example:

1) Are you using a dedicated network segment?

2) Is I/O to the LUN getting routed?

3) Do you use switches that share buffers among Port Groups?

4) Do you use Jumbo frames (end-to-end)

5) Possible driver issues on the host side. Is it possible the driver may not be responding to pause frame requests and continues to fwd packets which are dropped by the switch because of buffer overflow/overrun which requests a re-send of lost packets. Have you looked the switch port logs?

5) Have you try going Direct connect to the array Port to eliminate possible networking issues?

reinoud7 · ‎2008-04-02

Just to give some figures:

we use a FAS3020c as our central log filer: all our databases use this system for there logs. Depending on the database type, we use iSCSI (ms SQL) or FCP (Sybase).

With our sybase, our response time (write) is less than 0.6 ms (between 0.5 and 0.6 ms). For iSCSI, we see a response time of 0.6 ms.

Conclusion: we see almost the same response time for both environments.

When you need throuput, than FCP has an advantage if you can't use 10 GbE for iSCSI.In the near future the price of an 10 GbE will drop and then you can use this more easy and will have a higher bandwith than FCP.

nikhilm · ‎2008-04-02

Are you using Hardware iSCSI HBA or TOE Card to get the same perfomance as of 2/4 GBS FC card??

How does the host see iscsi lun , is this same c0t0d0 format or in different format ??

triantos · ‎2008-04-02

iSCSI performance is one of the most misunderstood aspects of the protocol. Looking at it purely from a bandwidth perspective, Fibre Channel at 2/4Gbit certainly appears much faster than iSCSI at 1Gbit. However, there are two important terms that need to be defined: Bandwidth and Throughput

Bandwidth: The amount of data transferred over a specific time period. This is measured in KB/s, MB/s, GB/s

Throughput: The amount of work accomplished by the system over a specific time period. This is measured in IOPS (I/Os per second), TPS (transactions per second)

There is a significant difference between the two in that Throughput has varying I/O sizes which have a direct effect on Bandwidth. Consider an application that requires 5000 IOPS at a 4k block size. That translates to a bandwidth of 20MB/s. Now consider the same application but at a 64k size. That's a bandwidth of 320MB/s.

Naturally, as the I/O size increases the interconnect with the smaller bandwidth will become a bottleneck sooner than the interconnect with the larger one (iSCSI vs FC).

At small block random I/O (4k-8k) both protocols perform equally well with similar IOPs and latencies but as the I/O size increases one (iSCSI) gets affected more than the other (FC)

TOEs and iSCSI HBAs do not guarantee higher performance and that's not the reason to deploy them. In fact, for a lot of workloads the native SW intiators outperform the iSCSI HBAs. The reason these cards came to fruition was to offload TCP and iSCSI processing overhead from the CPU. In an already underutilized server they provide no benefit unless you want to boot and even then you still don't need them given that there are NICs out there that support IP SAN boot using native iscsi stacks.

reinoud7 · ‎2008-04-02

No we don't use iSCSI HBA's, we have test this in the past (iSCSI HBA) but the microsoft iSCSI initiator is a great peace of software (for me, almost the best what microsoft has produced so far ).

So the performance is the same and in some cases better than an hardware card. The only disadvantage of the software initiator is that you consume some cpu cycles. But in the latest hardware, the CPU is not your bottleneck (even in a vmware environment) but your amount of memory is. So it's no problem to consume some cycles for iSCSI.

I have no (l)unix experiance with iSCSI. We use iSCSI only with windows. But I assume that there is also no difference between iSCSI and FCP.

nikhilm · ‎2008-04-02

So the question remain here is, is there any difference between iscsi and fc lun from application point of view.

flannery · ‎2008-04-03

No, there is no difference between a LUN served up via iSCS and a LUN served up via FCP. The difference is only in how the server connects to the LUN, the LUN is the same. As a matter of fact, I can create a database (SQL Server or Oracle or any other) in a LUN which is connected to the server via iSCSI. If I want to change this to FCP, I shut down the database, unmap the LUN from the iSCSI connection and remap it via an FCP connection and restart the database. The database or application will have no idea the connection protocol has just changed.

A LUN is a LUN, you connect to the LUN via iSCSI of FCP but the LUN doesn't change depending on which connection you choose.

Mike Flannery

Database Field Technical Lead

NetApp

oommen · ‎2008-10-24

Mike,

Would it be the same case if I tried the same using LVM(logical volume manager) on an AIX host. I created a LUN and mapped(iSCSI), On AIX created volume group, JFS2 file system on the hdisk and created a small file. The I re-mapped the LUN using FCP but somehow I cannot access the file system itself. I can send the sequence of steps what I performed on the AIX host. Not sure what I am doing wrong here. Much appreciate your help.

Bobby.

reinoud7 · ‎2008-04-03

Like Mike already explained: no there's no difference.

Because in most applications, respons time is much more important than throughput, we choose for almost all our applications for iSCSI. No issues with compatibility matrices, no expensive FC-switches ports, no HBA's, ... Today, we have more than 150 iSCSI hosts connect to our NetApp filers.

We use also FCP, but this is a very small FC-san (3 hosts). The raison for this is very simple: at that time, there was no support for the combination Sybase-Sun-iSCSI. Because it's our most important data and application, we decide to choose for the save option: FCP.

With 10 GbE at a lower price, the change is great that we remove the FC-san in the future.

nikhilm · ‎2008-04-03

I can see one difference in the lun from iscsi to fc. (see below o/p).

In FC lun target id is the filer port WWN where iscsi lun has target id t2. Hope this doesn't make a much difference to the application.

bash-3.00# /opt/NTAP/SANToolkit/bin/sanlun lun show

filer: lun-pathname device filename adapter protocol lun size lun state

kalinga: /vol/c000000182/lun0 /dev/rdsk/c1t500A09859637FFCBd0s2 qlc0 FCP 15g (16106127360) GOOD

filer960: /vol/nikhilVol/iscsilun4_0 /dev/rdsk/c5t2d0s2 qla4xxx0 iSCSI 1g (1073741824) GOOD

Yes 10GB ethernet is an great advantage over FC 2/4GBs.

But what about security in iscsi networks, FC is a dedicated network and more secure the LAN/WAN netowrks.

Does IPSec and VLAN for iscsi provide the same level of security and encryption, as in the FC network. ??

reinoud7 · ‎2008-04-03

I like the security discussions

1. If you think that this is more secure, you can alway get a complete separate iSCSI network with separate switches and with no link to your corporate network. I find this no good practice and I see this with some colleagues for two not technical raisons:

the storage admin's don't like the network guys and they want to have everything under control.
some stupid compliance rule that mentioned that your storage network could not connected with the corporate network.

2. Authentication and indentification is a part of the iSCSI protocol, so that's no problem

3. VLAN's is indeed a perfect solution for your ip-storage network: in a normal situation, this is just perfect. That's what we use in our environment.

I have no experiance with encryption of iSCSI trafic: in our case, iSCSI is only available in our datacenter, so I don't see the need to encrypt this in my case.

triantos · ‎2008-04-03

The difference you are seeing the LUNs is because of the driver. In your FC implementation you are using SUN's Leadville stack with the qlc driver which uses the array's WWN in the device file (t500A09859637FFCB500). Same gows for Emulex cards.

Had you use Qlogic's fibre channel qla driver then you'd have seen the device in the format your iSCSI HBA is showining it.

Prior to SUN introducing their native Fibre Channel stack, all devices appeared in the format your Qlogic iSCSI HBA is showing.

Yes...Fibre Channel is indeed an isolated network but a lot of folks fail to realize that breaks occur on the host which is connected to an...IP network. So the fact that you have a dedicated FC Network means little if the host has a wide open door...

You can also use a physically dedicated and isolated network with iSCSI and use CHAP in a two way authernication method.

BTW...FC provides DH-CHAP for initiator to switch, switch to array authentication but I know of no array that implements it today. What that means is that the host is still the weakest link...

So the bottom line is, unless security at the host level is not tightly implemented, it doesn't matter how you access the data on the array and how physically secure the backend network is.