Cisco Catalyst LACP-based port config for HyperV NIC load balancing

10 Gbit/s switches are slowly becoming affordable but still I see scenario’s where HyperV servers are disclosed via gigabit. To get enough bandwith to run a lot of machines, get your SAN storage traffic across and do live migrations, you need link aggregation to get multi-gigabit speeds.

There are a few mechanisms available in HyperV to use multiple NICs for load-balancing or failover scenarios. If your HyperV 2012 servers are attached to Cisco switches, then one of the most interesting (i.m.h.o.) is the use of LACP and transportports.

LACP or “Link Aggregation Control Protocol” allowes traffic to be routed to and from any port in a channel-group, basicly a group of switchports teamed together as a single link. If you’d like to know more about LACP then there are a lot of great articles on the internet (see “related posts & links” at the bottom).

TransportPorts is the method used for deciding how to effectively distribute outbound traffic across ports in the channel from the perspective of HyperV. Load-balancing requires a hash to be generated and each port in a LACP-portchannel gets part of the hashspace assigned. The hash-generation can be done on various parameters: source IP, destination IP, source port, destination port, VM MAC address. When using “transportports” the hash is generated by source IP + source port + destination IP + destination port. This has a major advantage over MAC-based load balancing that 2 connections to or from the same VM can be sent across different switchports.

HyperV uplink port profile

HyperV uplink port profile

From the point of view of HyperV the config is quite easy:

  • Open SCVMM and go to Fabric – Networking – Native Port Profiles.
  • Create an uplink port profile with
    • Load balancing algorithm = TransportPorts
    • Teaming mode: Lacp
  • Open the hyperv host config and go to Virtual Switches.
    • Add the desired physical network adapters to the virtual switch and choose to apply the newly created uplink port profile to them.
HyperV hardware network settings

HyperV hardware network settings

VMM will automatically push the right teaming config on the adapters and if you go to the server manager, select your HyperV server and choose to open the NIC teaming management console, then you will see the adapters registering as LACP capable as soon as your Cisco switch config is done. The renewed HyperV (or Windows 2012) network stack will auto-negotiate the correct LACP-settings and adapt to be compatible with the switch.

Next we need to push the right config on our Cisco Catalyst switches. An LACP port channel can be pushed across multiple stacked switches to achieve redundancy at hardware level. Just make sure you put half of your HyperV NICs on switch A and the other half on switch B to account for a single switch failure.

(TIP: redundancy applies to every aspect of your stack. Most admins are used to using dual power supplies but make sure you also use more than 1 network adapter card: f.e. using 2 dual-port cards is better than 1 quad-port card. If one card fails then you don’t loose the entire host)

First let’s define our VLANs

vlan 2301
name VL2301_test1
!
vlan 2305
name VL2305_test2
!

Next define a port-channel that will be used on the switchports and add the necessary VLANs

interface Port-channel12
description HV-host1 – TEAM
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 2301,2305,2320,2320-2335,2340,2350,2360,2370,2380,2381
switchport trunk allowed vlan add 2385-2390
switchport mode trunk
!

Put your switchports into this port-channel

interface GigabitEthernet1/0/12
description HV-host1 N1
channel-group 26 mode active
no shutdown
!
interface GigabitEthernet2/0/12
description HV-host1 N2
channel-group 26 mode active
no shutdown
!
interface GigabitEthernet1/0/13
description HV-host1 N3
channel-group 26 mode active
no shutdown
!
interface GigabitEthernet2/0/13
description HV-host1 N4
channel-group 26 mode active
no shutdown
!

When you take a look at the config via show run, you will see that the interface has been adapted and every bit of config from the etherchannel has been pushed on the ethernet interface except the description which stays “personal”.

interface GigabitEthernet1/0/12
description HV-host1 N1
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 2301,2305,2320,2320-2335,2340,2350,2360,2370,2380,2381
switchport trunk allowed vlan add 2385-2390
switchport mode trunk
channel-group 26 mode active
!

This has a nice advantage: put something new on the etherchannel (like adding a VLAN) and all interfaces in the channel-group will be getting the same config.

Next you can start putting virtual networks on top of the switch. In this case I’ve done isolation through VLAN-Ids but it would be possible to create only a few networks for your major services (cluster communication, live migration, frontend) and do the rest through NVGRE network virtualization. Beware however that the Microsoft implementation of network virtualization is not yet an accepted standard as other parties are working on similar technologies like VXLAN (vmware and commercial partners), OpenFlow (opensource community with support of some commercial partners). But we’ll certainly see some interesting developements in that area in a few years so I’m certain I’ll come back to that in the near future.

—————————————————————-

Was this post helpfull? Please help me keep this blog free and add more professional look. Any tiny donation can be made via Bitcoin so I can use it directly to upgrade my WordPress account.

My BTC address: 1DJF1TuJbWcL37tSf3iKP7TJFzTK7CpFQ4

Thanks a lot!

—————————————————————-

Related posts and/or external links:

http://www.cisco.com/web/techdoc/dc/reference/cli/nxos/commands/l2/channel-group__Ethernet.html

http://en.wikipedia.org/wiki/Link_Aggregation_Control_Protocol#Link_Aggregation_Control_Protocol

http://en.wikipedia.org/wiki/NVGRE

http://en.wikipedia.org/wiki/Virtual_Extensible_LAN

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: