There are 3 main approches for vlan tagging - External Swich Tagging (EST), Virtual Switch Tagging (VST), and Virtual Guest Tagging (VGT).
External Swich Tagging
With this approach ESX host doesn't see any vlan tags. All of them are stripped off by external physical switch and then traffic is sent to corresponding physical port. Therefore, you will need one vmnic per vlan from the ESXi perspective.
I really don't know in which situations such config might be useful.
Virtual Switch Tagging
This is the most popular way of connecting vSphere to physical network. All traffic down to the vSwitch is tagged with vlan id. It is responsibility of vSwitch now to strip off the vlan tag and send packet to virtual machine in corresponding port group. Thereby, you can run all vlans over one vmnic, but I guess you would like to have another vmnic for redundancy. The same logic is implemented to the packet travelling from virtual machine to physical infrastructure. Packet is delivered to vSwitch and before it is sent to physical switch the packet is tagged with vlan id according to the port group memebership of originating virtual machine.
Virtual Guest Tagging
In some occasions you might need to deliver traffic with vlan tags directly to VM and let your VM decide what to do with it. To achieve such functionality you need to assign this VM to port group with vlan id 4095 and configure Guest OS Nics with needed vlans. Interresingly, as soon as you put vlan id 4095 it is automatically changed to All (4095).
I didn't pay too much attention to reading about EST and VGT for the simple reasons it is not used at all in our vSphere farms and I don't want to spend time on something I will never have hands on experience with. However, when I questioned myself how I can sniff traffc on vSwitch google lead me directly to vlan 4095.
Here are the two popular use cases for vlan 4095:
1. Traffic sniffing - from time to time you face networking problems and woud like to use some sniffing tool (or network protocol analyzer in case of Wireshark) to see what is going on behind the scene. Usually, you could enable promiscious mode on specific port group and use one of the VM in this port group to listen to the traffic. However, this leads to additional security risks as Promiscious mode lets all VMs of this port group to detect all frames passed on the vSwitch in allowed vlan. This also doesn't let you to sniff traffic from different vlans simultaneously. That's why you would prefer to use special port group with vlan 4095 and Promiscious mode enabled, where you can connect virtual adapter of VM you will use for traffic sniffing.
2. IDS - Another good use case of vlan 4095 is to provide your virutal IDS with possibility to inspect all vSwich traffic. According to "VMware VSphere and Virtual Infrastructure Security: Securing the Virtual Environment" book this is quite a common scenario for virtual IDS placement.
If you know more situations where vlan 4095 can be useful feel free to share in comments. I really like to learn vSphere features with real life examples.
If you find this post useful please share it with any of the buttons below.
Great write up, thanks!
ReplyDeleteThanks...it was very helpful...
ReplyDeleteMan, you explained this topic in a way I could have never imagined. Thanks!
ReplyDeleteIn the 'vmxnet3 Ethernet Adapter Properties' image, what does that value of '10' mean here?
isn't it supposed to be 4095?
Ignore what I said, I didn't get enough sleep that night! :)
ReplyDeleteJust wanted to highlight a use case for External Switch Tagging. Where all your virtual machines belong to a single VLAN (e.g. vlan 200), there is no need for the vswitch to tag/untag packets. Just use a single default port group with no VLAN ID defined.
ReplyDeleteYou simply configure the port(s) on the physical switch as an access port (e.g. switchport access vlan 200). The switch takes care of all the tagging.
I've seen the above configured in a small production environment of less than 100 VMs all belonging to the same VLAN.
I'm just finishing off a blog post on intervlan routing in a virtual environment. It should be ready soon.
Thanks for feedback, gents.
ReplyDeleteMohammed, even though there are only 10 virtual machines I still prefer to avoid EST. If I ever need to provide one more VLAN to vSwitch I will have to reconfigure my physical switches.
VST provides all flexibility you may need at no cost. So, I don't really see why one wouldn't use it.
We have a situation where we VLAN off hotel rooms in order to segregate hotel guests on our network. e.g. Room100 might be VLAN 10, Room 101 VLAN 11, etc.
ReplyDeleteThis means we have a requirement to support several hundred VLANS on the same VM. We're trialing ESX as an option to host our gateway so that we can have more control of the host remotely and replace the gateway as the software & OS is updated.
The issue we are currently facing is the VLAN traffic gets tagged, handed to the Virtual Switch, given to the VM in the correct VLAN leaves the VM (tagged in the same VLAN) but never makes it through the virtual switch on the return trip. I suspect its something simple and we've missed it.
Virtual switch is set to VLAN 4095 with all the security settings like set to accept however, no return traffic yet. Would appreciate any help.
how did you fix the problem
Deleteapt-get install ethtool
Deleteethtool -K eth0 rxvlan off
Many thanks for that aricle - it was very helpfull!
ReplyDeleteEverything looks so clear now :)
Just one remark: It would be perfect if you could attach simple drawing for better understanding.
We use a port group with VLAN ID 4095 assigned to it for our client deployments that use virtual firewalls (Linux). The "internal" NIC of the firewall is assigned to that portgroup, and then VLAN interfaces created on the firewall (e.g., "eth1.1", "eth1.2", "eth1.10", etc.)
ReplyDeleteThe one down side I see is that you get all the packets turning up on that interface there is no filtering done on the vswitch....
ReplyDeletehow to set vlan ID inside VM after setting vmswitch port group to 4095
ReplyDelete