Load distribution over EtherChannels

I recently set myself a personal goal of taking the CCNP Switch exam. This being an exam focused on the Layer 2 technologies, amongst the topics approached is, of course, the EtherChannel. One of the things that is new in addition to what we already know about EtherChannels from CCNA is the way the load balancing is made over the physical links.

I’m not a mathematician but usually 1+1 equals 2. Well this is not quite the case with EtherChannels. At first, when we learn about EtherChannels we are tempted to assume that if we’ll aggregate two links of a given bandwidth, the capacity of the resulted Port-Channel will be twice the bandwidth of the physical ports. Or, one may think that if two links are being aggregated, when traffic will be sent across the EtherChannel the data will be shared 50/50 between the physical links.

Well, as i stated above, this is not quite the case and this is in direct relation with the load-balancing logic of the EtherChannel. The load share across the physical links is done by a per flow basis, not per packet or per frame.

What does this mean ? This means that if a communication starts between two hosts, a physical link from the EtherChannel will be chosen and it will be dedicated to that discution. To put it more simple, if we have link 1 and 2 being aggregated, the packets between host A and host B will only be sent over link 1 or 2, depending on which one will be chosen. But they will never go over both links 1 and 2 between hosts A and B.

When a link needs to be chosen, a Cisco propritary hash algorithm is run, which delivers a value from 0 to 7 for a maximum of 8 links in the EtherChannel. This values are asignated to the links in the EtherChannel in order to determine which flow goes on which link.

The algorithm can use any of the following: Source or destination IP address, source or destination MAC address, or a combination of both. Some switch models can also use source or destination TCP/UDP ports. Depending on the method configured, the switch may chose the hash of the last low-order bits, or it can perform an exclusive OR operation (XOR). The only time when XOR is involved is when both source and address are used in the process.  

I used some fancy words like last low-order bits, XOR, but let's see what do they mean. I will try to put it more simply by using two examples.

What are last low-order bits ?
Let’s say we have chosen to perform load balancing across a two link EtherChannel, based on the source IP address. In order to identify the links we need two values, because there are two links involved. This means that we will need one bit, who can be 1 or 0, giving us the required options. But where do we find that one bit ?

I stated above that we will chose to perform load balancing, based on the source IP address. Let’s assume a host with the IP address of 192.168.0.1 wants to send traffic over the EtherChannel. In order to determine on what physical link that flow will go, we will convert the last octet of the source IP address into binary. After this operation is done, from right to left we chose one bit. Depending on the value of that single bit, the Switch decides on what link from the EtherChannel to use.

Ex. Sending host : 192.168.0.1 --- choosing the last low-order bit(s) ---> 0000000[1]
Ex. Sending host : 192.168.0.2 --- choosing the last low-order bit(s) ---> 0000001[0]

Below, are the required number of low-order bits based on the number of links involved in the EtherChannel.

2 links --------------- 1 bit --------------- Values (decimal): 0,1

4 links --------------- 2 bits -------------- Values (decimal): 0,1,2,3

8 links --------------- 3 bits -------------- Values (decimal): 0,1,2,3,4,5,6,7

A legitimate question is what happens when the number of links being aggregated is 3, 5 or 7 ? Cisco does not recomend that because load-balancing issues can occur.

Ok, the second fancy term used was exclusive OR operation, or XOR for short. What is that ?

This is just a simple binary operation where 0 and 0, or 1 and 1 result in 0, and where 1 and 0 combinations result in a value of 1. To put it more simple, i will again use an example.

Let’s say we have a comunication between two hosts, over a four link EtherChannel, with 192.168.0.1 being the source and the 172.16.0.3 being the destination. As above, we will convert the last octets into binary and chose the last two bits from right to left, because we need four values (four links in the EtherChannel).

Source --------------- 192.168.0.1 --------------- The last octet in binary: 000000[01]

Destination ----------172.16.0.3 ----------------- The last octet in binary: 000000[11]

By performing the XOR on the above values, we will get a binary value of 10 or 2 in decimal.
1 and 0 result in a 1, and 1 and 1 result in a value of… 0.

And this is pretty much it.
In order to find more about EtherChannels, you can follow this link. You will also find there some useful CLI commands to test and verify the EtherChannels.

GNS3 Laboratory

I have designed a simple GNS3 lab to observe how the load-balancing works across an EtherChannel.

To emulate the switch behavior i used two Cisco 3725 routers with a NM-16ESW module. After downloading the lab, you will need to start a WireShark and filter by ICMP on both links from the EtherChannel. When this is done open a console on R1 and R2.

Use the show run int fast 0/0 command on both routers to see the IP configurations. Generate some traffic by sending pings from one to the other, and observe in the Wireshark captures how the links are being used.

Topology

Download the lab.

Since this is new ground for me also, i’ll appreciate any feedback which can improve this material.
I hope you’ll find this useful. Take care!