Subscribe
Accepted Solution

New Cluster Nodes Not Showing in NetApp DSM for MPIO

I've just added two new nodes to our production cluster. I've added two new iSCSI LIFs on the nodes and added those LIFs to the port group. I've used SnapDrive to add connections to the two new iSCSI LIFs on the first of many Windows hosts. After doing this, I checked the Data OnTAP DSM for MPIO software to be sure it was showing 4 nodes per LUN. It is not. It is still just showing two paths. Any ideas as to what I am missing?

Re: New Cluster Nodes Not Showing in NetApp DSM for MPIO

Assuming you are running cDoT 8.3 or higher, you are likely encountering the new "Selective LUN Mapping" feature.  This feature was added in cDoT 8.3 to address the multiplication of paths across large clusters that can impact server based total and per LUN path limits.

 

From the SAN Administration Guide...

 

Selective LUN Map

Selective LUN Map (SLM) reduces the number of paths from the host to the LUN. With SLM, when a new LUN map is created, the LUN is accessible only through paths on the node owning the LUN and its HA partner.

SLM enables management of a single igroup per host and also supports nondisruptive LUN move operations that do not require portset manipulation or LUN remapping.

Portsets can be used with SLM just as in previous versions of Data ONTAP to further restrict access of certain targets to certain initiators . When using SLM with portsets, LUNs will be accessible on the set of LIFs in the portset on the node that owns the LUN and on that node's HA partner.

Beginning with Data ONTAP 8.3 SLM is enabled by default on all new LUN maps. For LUNs created prior to Data ONTAP 8.3, you can manually apply SLM by using the lun mapping remove-reporting-nodes command to remove the LUN reporting nodes and restrict LUN access to the LUN owning node and its HA partner.

 

 

 

I ran into the same situation myself when adding nodes 5 and 6 to a cluster, moving some LUNs there, and then not seeing direct paths.  Mine was FCP, but since this applies at the LUN mapping level it would be just as applicable to iSCSI as well.

 

The commands of interest to you are:

 

lun mapping show -field reporting-nodes -vserver <SVM> -path <lun-path>

lun mapping add-reporting-nodes ...

lun mapping remove-reporting-nodes ...

 

SLM provides for very specific path control and in conjunction with portsets can limit path proliferation especially as the number of potential ports increases and the number of allowed hosts in a SAN cluster increases with refreshed hardware and ONTAP 9.1.

 

 

Hope this helps.

 

Bob Greenwald

Senior Systems Engineer | cStor

NCIE SAN ONTAP, Data Protection

 

 

Kudos and accepted solutions are always accepted.

 

Re: New Cluster Nodes Not Showing in NetApp DSM for MPIO

Very helpful! Thank you very much Bob.

Re: New Cluster Nodes Not Showing in NetApp DSM for MPIO

Bob, one more question: from what I can tell the -aggregate and -volume options are optional. I simply want each LUN in a given SVM to have access to two new nodes in the cluster, regardless of aggregate. Am I correct that I can leave these switches off? Also if I enter a common path name with a wildcard for the -lun option, will that work? For example:

 

lun mapping add-reporting-nodes -vserver <vserver> -path <Server_Name>_* -igroup <Igroup-Name>

Re: New Cluster Nodes Not Showing in NetApp DSM for MPIO

 

Your comand fragment is enough to identify the particular LUN/igroup combinations of interest.  You'll need one of the four optional parameters to select which nodes to add to the list...

 

-local-nodes [ true ]   adds the nodes where the LUN currently exists to the reported path map.

 

-destination-aggregate <aggregate name>    adds the nodes where the listed aggregate lives.  Typically used before a volume would be moved to a new aggregate.  Could also be used with the name of an aggregate on your new nodes - nothing says the volume has to be moved there.

 

-destination-volume <volume name>   adds the nodes where the listed volume lives.  Typically used before a LUN is moved to a new volume (lun move start ...).  Could also be used with the name of a volume on your new nodes - nothing says the LUN has to be moved there.

 

-all   adds all the nodes in the cluster.  This option is available at "advanced" privilege level (set -priv adv).  Gets a bit scary in large clusters.

 

 

One other thought:  remember for path management the only paths you really need are the ones to the HA pair where the aggregate/volume/LUN live.  If one node fails, the aggregate flips over to the other node and you still have direct paths.  If both nodes fail the aggregate is offline.  Having non-optimized paths through other nodes in the cluster is only needed if you suspect that all the iSCSI data paths to the owning HA pair could fail at the same time.  But, if your network/system design is such that you might lose all those paths at the same time, would it also take down the paths you just added for the new nodes?  If so, having the additional paths does not add protection but does add overhead at both ends (small, but adds up at scale).

 

 

Hope this helps.

 

Bob Greenwald

Senior Systems Engineer | cStor

NCIE SAN ONTAP, Data Protection

 

 

Kudos and accepted solutions are always appreciated.

 

 

 

Re: New Cluster Nodes Not Showing in NetApp DSM for MPIO

Thank you very much Bob, and yes it definitely helps. The reason for this design is that we are planning to move all of our iSCSI volumes/LUNs to the two new nodes in the cluster (they are AFF nodes).

 

Based on your description of the four options, it sounds like my options are "destination-aggregate" or "all". I don't really understand "local-nodes" - wouldn't the nodes that currently contain the LUNs already be "reporting nodes"? Mine are. Do you need to "add" nodes that are already applied when you are making this kind of change? My concern about the "all" option is I'm concerned about the EXISTING reporting nodes - I don't want to adversely affect them somehow.

 

Here are a couple possible ways for me to do this based on my understanding. Does this look correct?

 

lun mapping add-reporting-nodes -vserver <vserver> -path <Server_Name>_* -igroup <Igroup-Name> -destination-aggregate <aggregate_on_node_3>

lun mapping add-reporting-nodes -vserver <vserver> -path <Server_Name>_* -igroup <Igroup-Name> -destination-aggregate <aggregate_on_node_4>

 

lun mapping add-reporting-nodes -vserver <vserver> -path <Server_Name>_* -igroup <Igroup-Name> -all

Re: New Cluster Nodes Not Showing in NetApp DSM for MPIO

[ Edited ]

LUN move is actually covered pretty well in documentation (see SAN Administration Guide, Modifying SLM reporting nodes). When you move LUN to new aggregate that is hosted on different nodes, you need to add these nodes to the set of reporting nodes. Using -destination-aggregate eliminates need to know these nodes' names directly - you usually know destination aggregate/volume already. As long as you have just 4 nodes in total, this is entirely equivalent to using "all". But it makes difference if you have large cluster.

 

After move is complete you may remove old nodes using remove-reporting-node -remote-nodes.

Re: New Cluster Nodes Not Showing in NetApp DSM for MPIO

Thanks aborzenkov. Do you or Bob know if there's a preferred / recommended method? I'm slightly concerned about using "all" since that includes the existing reporting nodes. Do either of you know if "all" only affects nodes that aren't currently applied?

Re: New Cluster Nodes Not Showing in NetApp DSM for MPIO

In particular, the description of "all" in the man pages is:

 

------------------------------------------------

Set the LUN mapping to report on all nodes in preparation for a revert to a previous version of Data ONTAP.

------------------------------------------------

 

That doesn't seem right. I prefer "all" because it's simple, if indeed it is safe to use.

 

If I use "destination-aggregate", does it truly matter which aggregate I choose as long as it's on the applicable node? In other words, could I choose an aggregate on node 3 for all LUNs but actually move some to node 3 and some to node 4?

 

Re: New Cluster Nodes Not Showing in NetApp DSM for MPIO

One last question re: syntax:

 

For the LUN path, do I need to do this (<servername>* are the volumes, <servername>*.lun are the LUNs):

 

-path /vol/<servername>*/<servername>*.lun

 

or will this work:

 

-path /vol/<servername>*