Broadcast Domains and Failover Groups

[tyrone_owen_1]

Hi,

I'm hoping someone can help me understand the benefit of the setup below. It was set up by a third party.

In this config, there are no overlapping IP address and there are 2 xSVMs which are members of the 'Default' IPspace

• The Default Broadcast Domain (BD) and Failover Group (FG) contains offline ports, so a redundant configuration
• The 'Cluster-Management' BD and FG contains both the physical and VLAN ports
• There is a dedicated BD and FG 'Data' for all of the physical ports in use
• The 'SVM%-Data' BD and FGs contain only the VLAN ports

Vserver	Group	Targets
------	--------------------------	--------------------------------------------
CLUSTER
	Cluster-Management
		N01:e0M,
		N01:e0e-1111,
		N01:e0f-1111,
		N01:e0g-1111,
		N01:e0h-1111,
		N02:e0M,
		N02:e0e-1111,
		N02:e0f-1111,
		N02:e0g-1111,
		N02:e0h-1111
	Data
		N01:e0e,
		N01:e0f,
		N01:e0g,
		N01:e0h,
		N02:e0e,
		N02:e0f,
		N02:e0g,
		N02:e0h
	Default
		N01:e0c,
		N01:e0d,
		N02:e0c,
		N02:e0d
	SVM2-Data
		N01:e0e-2222,
		N01:e0f-2222,
		N01:e0g-2222,
		N01:e0h-2222,
		N02:e0e-2222,
		N02:e0f-2222,
		N02:e0g-2222,
		N02:e0h-2222
	SVM1-Data
		N01:e0e-3333,
		N01:e0f-3333,
		N01:e0g-3333,
		N01:e0h-3333,
		N02:e0e-3333,
		N02:e0f-3333,
		N02:e0g-3333,
		N02:e0h-3333
	SVM2-Management
		N01:e0e-4444,
		N01:e0f-4444,
		N01:e0g-4444,
		N01:e0h-4444,
		N02:e0e-4444,
		N02:e0f-4444,
		N02:e0g-4444,
		N02:e0h-4444
	SVM1-Management
		N01:e0e-5555,
		N01:e0f-5555,
		N01:e0g-5555,
		N01:e0h-5555,
		N02:e0e-5555,
		N02:e0f-5555,
		N02:e0g-5555,
		N02:e0h-5555

1. Is there any reason to deviate from using the 'Default' Broadcast Domain and Failover Group?
2. Should the asscoated VLAN and physical ports that the LIFS are allowed to fail over to be contained in the same BD/FG?

Thanks

[GidonMarcus]

Hi.

1. Is there any reason to deviate from using the 'Default' Broadcast Domain and Failover Group?
   1. No - the Default BD and FG used by most of the customers. 
2. Should the asscoated VLAN and physical ports that the LIFS are allowed to fail over to be contained in the same BD/FG?
   1. No - there's no reason the physical group to be in any BD/FG (unless you for some reason using the native Vlan) . And only a single VLAN have to be present in any DB/FG

Having said all the  -your configuration doesn't seem to be "wrong". you can have extra BD/FG for the physical ports... (mainly in case thy have native vlan). and you are allowed not to use the default DB/FG if you don't wish to.

Taking the fact you are using failover group, i assume this is not an iSCSI setup, right?

if so, it is a bit alarming that you don't seem to be using LACP at all - instead - you trust ONTAP to detect the port is down and move the LIFs over. i think LACP give you better/faster tolerance and more throughput + workload spreading across the switches.  the failover between the ports/switches will also be more seamless to some clients if it's done on the 2nd OSI layer (LACP) rather the 3rd (LIF failvoer).

[TMAC_CTG]

Adding my take here.

Broadcast-domains should always contain ports that are similar and can always fail over to each other. They should NOT contain any unused ports or ports on different VLANs (as already indicated). Why? in a LIF failover scenario, if configured, can go to any port in the Broadcast Domain. I have had customer accidentally leave everything in the same Broadcast Domain and when the switch failed, the port moved to another on a differetn VLAN (so link was good, networking, not so much) and all communications stopped.

When you configure a LIF to have a "failover-group" (which is a leftover term from before Clustered ONTAP 8.3 ), and you have the LIF failover policy set to "broadcast-domain-wide" then all ports in the broadcast-domain *must* be in the same VLAN.

Depending on the platform, I might do something like have e0M and e0j in the same Broadcast-domain. e0M is active and e0j is hooked up and on the same VLAN (access-port on the switch) but just not in active use.

For the BD groups, when defining VLANs, I try to keep all the ports in the BD with the same tagged VLAN. In the example, there is reference to e0M and e0x-1111 in the same BD. Not ideal from the standpoint that Active-IQ and even Config Advisor will through a caution/warning since the LIF failover will switch from an access-port to a tagged vlan. The other examples appear to be fine (expect I would merge all four ports into an LACP channel and then tag the VLAN on the ifgrp)

With all that said, if you have a 100% flat network and do not use any VLAN tags, then you can probably get by with the Default.

If you have multiple subnets and/or you have VLANs, if you want failover to work without an issue, you MUST define a Broadcast-Domain for each Network (i.e. each subnet, like 192.168.1.0/24 and 192.168.2.0/24). Additionaly, if you are doing things like VMware over NFS or iSCSI, it is usually best to use an MTU of 9000 (as long as the network infrastructure supports it!) and you can define the MTU at the broadcast domain.

[tyrone_owen_1]

Thanks. It's not iSCSI.

The system was set up by a NetApp reseller.

I think the BD/FG per VLAN is required. Say for example, you had a single BD/FG containing VLAN ports from two separate VLANs - they wouldn't be able to fail over to each other. As a consequence, you would need two DB/FGs, one for each VLAN.

Is that correct?

[GidonMarcus]

For the last part - Exactly.

Was there a good reason not to use LACP? Anything from the network design side ?

See page 44 for what i believe is the latest best practice in this field:

https://www.netapp.com/us/media/tr-4182.pdf