Multipath iSCSI boot LUN

MarktheSpark — Thu, 13 Feb 2025 14:34:49 GMT

Hello all,

I've been struggling to setup multipathing on an iSCSI boot LUN so after many days banging my head against a brick wall, I'm hoping somebody can point me in the right direction.

Our development setup is as follows:

FAS8200 running ONTAP 9.13.1P13
Cisco UCS X210c M7 blade server
Oracle Linux 8.10 running 5.15.0-209.161.7.1.el8uek.x86_64 kernel

Installation was fine, once I added ip=ibft to the boot command. I have setup two interfaces on the host and two on the FAS8200 vserver to cater for iSCSI and have established four paths:

[root@vmhost-dev-b-02 ~]# iscsiadm --mode session
tcp: [1] 10.31.5.101:3260,1026 iqn.1992-08.com.netapp:sn.c052244ddc6a11eeb08a00a098d45a03:vs.7 (non-flash)
tcp: [2] 10.31.6.102:3260,1031 iqn.1992-08.com.netapp:sn.c052244ddc6a11eeb08a00a098d45a03:vs.7 (non-flash)
tcp: [3] 10.31.6.101:3260,1030 iqn.1992-08.com.netapp:sn.c052244ddc6a11eeb08a00a098d45a03:vs.7 (non-flash)
tcp: [4] 10.31.5.102:3260,1027 iqn.1992-08.com.netapp:sn.c052244ddc6a11eeb08a00a098d45a03:vs.7 (non-flash)

I can see those sessions are logged on from the filer:

fas8200a::> iscsi session show -vserver vsbidev1 -initiator-name iqn.2022-10.uk.ac.lboro:site-dev-b-iscsi-a:2
Tpgroup Initiator Initiator
Vserver Name TSIH Name ISID Alias
--------- ------- ---- ------------------------ --------- ---------------------
vsbidev1 vsbidev1-01-a 2 iqn.2022-10.uk.ac.lboro:site-dev-b-iscsi-a:2 00:02:3d:00:00:01 vmhost-dev-b-02.lboro.ac.uk
vsbidev1 vsbidev1-01-b 5 iqn.2022-10.uk.ac.lboro:site-dev-b-iscsi-a:2 00:02:3d:00:00:03 vmhost-dev-b-02.lboro.ac.uk
vsbidev1 vsbidev1-02-a 2 iqn.2022-10.uk.ac.lboro:site-dev-b-iscsi-a:2 00:02:3d:00:00:04 vmhost-dev-b-02.lboro.ac.uk
vsbidev1 vsbidev1-02-b 1 iqn.2022-10.uk.ac.lboro:site-dev-b-iscsi-a:2 00:02:3d:00:00:02 vmhost-dev-b-02.lboro.ac.uk
4 entries were displayed.

Using NetApp's Linux Host Utilities I can see the LUN and four paths:

[root@vmhost-dev-b-02 ~]# sanlun lun show
controller(7mode/E-Series)/ device host lun
vserver(cDOT/FlashRay) lun-pathname filename adapter protocol size product
----------------------------------------------------------------------------------------------------------
vsbidev1 /vol/vmhost_dev_b_02/rocky-dev-b-02 /dev/sdd host6 iSCSI 100g cDOT
vsbidev1 /vol/vmhost_dev_b_02/rocky-dev-b-02 /dev/sdc host5 iSCSI 100g cDOT
vsbidev1 /vol/vmhost_dev_b_02/rocky-dev-b-02 /dev/sdb host4 iSCSI 100g cDOT
vsbidev1 /vol/vmhost_dev_b_02/rocky-dev-b-02 /dev/sda host3 iSCSI 100g cDOT

But multipath refuses to see these devices as multipath devices:

[root@vmhost-dev-b-02 ~]# multipath -l -v 3
Feb 13 14:26:14 | set open fds limit to 4096/262144
Feb 13 14:26:14 | loading /lib64/multipath/libchecktur.so checker
Feb 13 14:26:14 | checker tur: message table size = 3
Feb 13 14:26:14 | loading /lib64/multipath/libprioconst.so prioritizer
Feb 13 14:26:14 | foreign library "nvme" loaded successfully
Feb 13 14:26:14 | sda: size = 209715200
Feb 13 14:26:14 | sda: vendor = NETAPP
Feb 13 14:26:14 | sda: product = LUN C-Mode
Feb 13 14:26:14 | sda: rev = 9131
Feb 13 14:26:14 | sda: h:b:t:l = 3:0:0:0
Feb 13 14:26:14 | sda: tgt_node_name = iqn.1992-08.com.netapp:sn.c052244ddc6a11eeb08a00a098d45a03:vs.7
Feb 13 14:26:14 | sdb: size = 209715200
Feb 13 14:26:14 | sdb: vendor = NETAPP
Feb 13 14:26:14 | sdb: product = LUN C-Mode
Feb 13 14:26:14 | sdb: rev = 9131
Feb 13 14:26:14 | sdb: h:b:t:l = 4:0:0:0
Feb 13 14:26:14 | sdb: tgt_node_name = iqn.1992-08.com.netapp:sn.c052244ddc6a11eeb08a00a098d45a03:vs.7
Feb 13 14:26:14 | sdc: size = 209715200
Feb 13 14:26:14 | sdc: vendor = NETAPP
Feb 13 14:26:14 | sdc: product = LUN C-Mode
Feb 13 14:26:14 | sdc: rev = 9131
Feb 13 14:26:14 | sdc: h:b:t:l = 5:0:0:0
Feb 13 14:26:14 | sdc: tgt_node_name = iqn.1992-08.com.netapp:sn.c052244ddc6a11eeb08a00a098d45a03:vs.7
Feb 13 14:26:14 | sdd: size = 209715200
Feb 13 14:26:14 | sdd: vendor = NETAPP
Feb 13 14:26:14 | sdd: product = LUN C-Mode
Feb 13 14:26:14 | sdd: rev = 9131
Feb 13 14:26:14 | sdd: h:b:t:l = 6:0:0:0
Feb 13 14:26:14 | sdd: tgt_node_name = iqn.1992-08.com.netapp:sn.c052244ddc6a11eeb08a00a098d45a03:vs.7
Feb 13 14:26:14 | dm-0: device node name blacklisted
Feb 13 14:26:14 | dm-1: device node name blacklisted
Feb 13 14:26:14 | dm-2: device node name blacklisted
===== paths list =====
uuid hcil dev dev_t pri dm_st chk_st vend/prod/rev dev_st
3:0:0:0 sda 8:0 -1 undef undef NETAPP,LUN C-Mode unknown
4:0:0:0 sdb 8:16 -1 undef undef NETAPP,LUN C-Mode unknown
5:0:0:0 sdc 8:32 -1 undef undef NETAPP,LUN C-Mode unknown
6:0:0:0 sdd 8:48 -1 undef undef NETAPP,LUN C-Mode unknown
Feb 13 14:26:14 | libdevmapper version 1.02.181-RHEL8 (2021-10-20)
Feb 13 14:26:14 | DM multipath kernel driver v1.14.0
Feb 13 14:26:14 | unloading const prioritizer
Feb 13 14:26:14 | unloading tur checker

I suspect the issue has something to do with the fact that root is sitting on this disk which is preventing is being changed. I've read suggestions that I need to enable multipathing at boot time and change the paths to the various partitions but I'm not sure if that will work or not. Any help or guidance would be gratefully received.

Regards,

Mark

Re: Multipath iSCSI boot LUN

AmitKerkar — Thu, 06 Mar 2025 15:45:46 GMT

faced similar issue decades ago.

have you updated the initramfs ?

echo "multipath" >> /etc/dracut.conf.d/multipath.conf

rebuilt the image and boot

it will load everything before mounting the root volume

Re: Multipath iSCSI boot LUN

ro-st — Thu, 19 Feb 2026 15:03:43 GMT

Does anybody have a solution for Proxmox/Debian how to activate multipath for the iSCSI-Boot LUN? And some more information how to exactly do that - I am not that familliar with modifiying the initramfs. Thank you!

Re: Multipath iSCSI boot LUN

MarktheSpark — Thu, 19 Feb 2026 16:12:01 GMT

Hi ro-st,

I created some extensive internal documentation for installing Debian on an iSCSI boot LUN before converting that to Proxmox. I think the bit you are looking for is the following, although this was done during installation so you may have adjust accordingly:

apt-get update

apt-get install multipath-tools

multipath -a /dev/sda #or whatever you boot LUN is

apt-get install multipath-tools-boot

update-initramfs -u

If multipathing hasn't worked then simply try again as follows

multipath -a /dev/sda #or whatever you boot LUN is

update-initramfs -u

reboot

Hope it helps,

Mark

Re: Multipath iSCSI boot LUN

ro-st — Tue, 24 Feb 2026 10:47:14 GMT

Hello Mark,

thank you for your reply. I got it working doing a Debug install with the pve9.1 ISO. Before GUI install, I installed and configured iSCSI and after GUI install before reboot I did a chroot and installed "multipath-tools multipath-tools-boot lsscsi" and configured the 2nd path with "multipath -a" as you described and finished with "update-grub && update-initramfs -u -k all". Multipath did then work after some finishing work on network and rescan and a second reboot.

Kind regards,

Robert

Re: Multipath iSCSI boot LUN

MarktheSpark — Tue, 24 Feb 2026 10:52:40 GMT

Hi Robert,

Glad to hear you got it working. I'm might revisit iSCSI boot since you have it working with the pve9.1 debug install. Thanks for the tip.

Regards,

Mark

Re: Multipath iSCSI boot LUN

ro-st — Tue, 24 Feb 2026 11:23:51 GMT

Most of the steps came from this post and worked at first try:
https://forum.proxmox.com/threads/install-pve-directly-on-iscsi-target.101750/post-438664

And then additionally the multipath thing.

topic Re: Multipath iSCSI boot LUN in Network and Storage Protocols

Multipath iSCSI boot LUN

Re: Multipath iSCSI boot LUN

Re: Multipath iSCSI boot LUN

Re: Multipath iSCSI boot LUN

Re: Multipath iSCSI boot LUN

Re: Multipath iSCSI boot LUN

Re: Multipath iSCSI boot LUN