Microsoft Virtualization Discussions
Microsoft Virtualization Discussions
Hi all,
I just had a look at "NetApp Storage Best Practices for Microsoft Virtualization" (http://www.netapp.com/us/library/technical-reports/tr-3702.html). The table about physical NICs requirement on page 8 in somewhat confusing to me - I think it relates to a config without NIC redundancy, so ideally twice as many ports are required. The following page talks about NIC teaming as being doable, but I assume I shouldn't team ports across different types of traffic, should I?
If above is correct, the staggering number of 14 physical ports appears to be needed for a fully blown Hyper-V implementation...
Any thoughts, comments, ideas?
Regards,
Radek
Solved! See The Solution
Radek,
Maybe I can provide some insight, as the author of TR-3702; I can explain how these best practices came about and offer supporting evidence. This is because most of our recommendations mirror those that Microsoft has provided in their technical documentation about Hyper-V. When I reference sections and page numbers in this reply, please assume (unless otherwise noted) that they are referring to content in TR-3702 v3.
On page 7, I begin to discuss the different types of network connectivity that is present in a Hyper-V server, if we assume all of the network connectivity represented by each of the bullets on page 7 is present in a Hyper-V environment – I am assuming that you have deployed more than one Hyper-V server and those Hyper-V servers are clustered together with the VMs (and their VHDs) deployed on Cluster Shared Volumes (CSVs, see page 32) over iSCSI connectivity, and that you are making use of Live Migration in the Hyper-V environment to move VMs between Hyper-V servers in the cluster. Although I feel I have explicitly discussed the reason for the types of network connectivity listed on page 7, let me briefly discuss it here too.
Based upon Microsoft’s recommendations, which NetApp follows and in some cases improves upon, the minimum number of network adapters ideal for a Hyper-V configuration, as I outlined in the opening paragraph of this reply, is 5 for a non-production deployment where redundancy is not a concern. Where redundancy is a concern, such as in production environments, then the minimum number is 7, because we’d want to use MPIO for iSCSI connectivity if present, and multiple networks adapters (likely teamed) for VM external connectivity. In the tables on page 8, I take into account the need for multiple physical network adapters for redundancy when using Clustered Hyper-V servers using live migration, CSVs, and configured with iSCSI – therefore, 7 or 8 network adapters is the minimum number of network adapters for a production Hyper-V environment. This assume that redundancy is not needed for the followin
- Hyper-V Management –although if failure occurs here, then there is little option to manage the Hyper-V host, therefore VMs would have to be migrated off remotely or the server would have to be powered down and VMs would incur an outage in the meantime. I have customers that use the network adapter for the private cluster communication as a backup.
- Windows Failover Cluster Private – if failure occurs here, Hyper-V may migrate VMs to another node in the cluster or provide a warning. Some customer use the network adapter configured for Hyper-V management as a backup.
- Live Migration – This is a bit of a risk, but most customers aren’t using any functionality to dynamically migrate the VMs between nodes yet, so the risk is minimized in those configurations. If a failure occurs here, the nodes would use the Windows Failover Cluster Private network as a backup, the only risk being that multiple Live Migrations could saturate the link and cause problems for the cluster as discussed above or the ability to do successful Live Migrations.
- Cluster Shared Volumes – This a probably the biggest risk of all, but just like the Live Migration connectivity, if a failure occurs here, the nodes would use the Windows Failover Cluster Private network as a backup. For more information on the possible risk, please see page 35 and section 4.3.3.4 on Dynamic I/O Redirection or “Understanding redirected I/O mode in CSV communication” here.
As far as NIC teaming goes, the recommendations in the industry ar as follows:
Honestly, although our recommendations are spelled out pretty clearly, for some of this connectivity it is bandwidth that is the primary concern that leads to these recommendations which were made primarily on the assumption that most customers will be using 1GbE network adapters. In the next version of TR-3702, I plan to add a few pages and make the distinction between recommended network configurations for those deploying with 1GbE connectivity and 10GbE connectivity. The issue is that most customers who deploy 10GbE will have a mix of that connectivity, because server will have 1GbE onboard in either 1 or 2 adapters, but customers may deploy mixes of Quad port 1GbE and 10GbE adapters in PCI slots.
Here are a few –off the record and example only – configurations; where all assume that you don’t have the minimum number of ports available or are using a mix of 1GbE and 10GbE connectivity, probably found in a 1U server that is limited to two PCI slots.
· Config #1
o Network Adapters Available:
§ Two 1GbE NICs onboard – NIC1 and NIC2
§ Four 1GbE NICs in PCI slots (Quad Port adapter) – NIC3, NIC4, NIC5, and NIC6
§ One 10GbE NICs in the 2nd PCI slot – NIC7
o As configured:
§ NIC1 – Hyper-V Management + Windows Failover Cluster Private
§ NIC2 – iSCSI #1
§ NIC3 – iSCSI #2
§ NIC4 – VMs
§ NIC5 – VMs
§ NIC6 – Windows Failover Cluster Private + Hyper-V Management
§ NIC7 – Cluster Shared Volumes (CSVs) + Live Migration
· Config #2
o Network Adapters Available:
§ Two 1GbE NICs onboard – NIC1 and NIC2
§ Four 1GbE NICs in PCI slots (Quad Port adapter) – NIC3, NIC4, NIC5, and NIC6
§ Two 10GbE NICs in the 2nd PCI slot – NIC7 and NIC8
o As configured:
§ NIC1 – Hyper-V Management + Windows Failover Cluster Private
§ NIC2 – iSCSI #1
§ NIC3 – iSCSI #2
§ NIC4 – VMs
§ NIC5 – VMs
§ NIC6 – Windows Failover Cluster Private + Hyper-V Management
§ NIC7 – Cluster Shared Volumes (CSVs)
§ NIC 8 – Live Migration
· Config #3
o Network Adapters Available:
§ Two 1GbE NICs onboard – NIC1 and NIC2
§ Four 1GbE NICs in PCI slot (Quad Port adapter) – NIC3, NIC4, NIC5, and NIC6
o As configured:
§ NIC1 – Hyper-V Management + Windows Failover Cluster Private
§ NIC2 – iSCSI #1
§ NIC3 – iSCSI #2
§ NIC4 – VMs
§ NIC5 – VMs
§ NIC6 – Cluster Shared Volumes (CSVs) + Live Migration
· Config #4
o Network Adapters Available:
§ Two 1GbE NICs onboard – NIC1 and NIC2
§ Two 10GbE NICs in PCI slot (Dual Port adapter) – NIC 3 and NIC4
o As configured:
§ NIC1 – Hyper-V Management + Windows Failover Cluster Private
§ NIC2 – iSCSI
§ NIC3 – VMs
§ NIC4 – Cluster Shared Volumes (CSVs) + Live Migration
If you are looking for more information on a recommended configuration based on the number of network ports that you do have available, I would be happy to discuss a possible configuration that best suits the environment. This is a conversation I often have with customers who are using blade servers for deploying Hyper-V or unique solutions such as HP’s Flex-10 – both of which fall outside of my off the record examples above. I can be contacted through the communities or via twitter @virtualizethis.