VMware Virtual SAN Network Design Guide v2.0 is just released

VMware has just released the document that covers network aspects of Virtual SAN design. 

Actually, the guide has been re-released as there was v1.0 before (hence the new one goes under v2.0), but as far as I am aware it was removed from the VMware web site due to some inaccuracies in it. So for a while people lacked validated design information on one of the key aspect of VSAN setup. I remember there were quite a few discussions on how to provide network redundancy and load-balancing for VSAN traffic and nobody could get a formal answer.

The guide is very comprehensive and even provides mutlicast configuration examples on Cisco and Brocade switches.

You can check the guide here

Check out the new VSAN 6.2 Hands-On-Lab

VMware has just released new "HOL-SDC-1608 What's New with Virtual SAN 6.2" Hands-On-Lab

It covers all new functionality the VMware brought in new VSAN 6.2, e.g. compression and deduplication on AF, new SPBM settings, new ESXCLI VSAN namespace, etc.

However, the HOL assumes you have basic knowledge of VSAN. For instance, networking design isn't covered here. 

Interestingly, the networking configuration in this HOL still contains 2  VSAN VMK interfaces whereas generally it is recommended to have only 1 VMK and provide HA by NIC teaming and Load Balancing with LACP.

Even though I have VSAN 6.2 deployed in my home lab I still skimmed through the VSAN 6.2 HOL and I can tell it is pretty useful as I have learnt something new.

You can find it here - VSAN 6.2 HOL

How to deploy VSAN 6.2 Witness Virtual Appliance to VMware Fusion 8.1

Since I moved to vSphere 6 U2 I started to plan my VSAN deployment. The problem I have is that there are only two physical servers in my home lab and I have no physical server to host VSAN Witness appliance. I am trying to make it run on my Synology DS415+ (crazy, but seems to be doable), but I need to upgrade RAM on Synology first.

As a temp solution I thought I could run VSAN 6.2 Witness on my Mac. William Lam has already posted a great post on how to Deploy and Run VSAN 6.1 Witness in VMware Workstation/Fusion, but I faced a small issue when following it. 

First it warned me that there is OVF specification and hardware compliance mismatch.  

And after that it failed to proceed with error "Line 821: Unexpected element 'Propery'"

Line 821 referred to the line we have to adjust in Step 3 as per William's guideline. I don't really know whether OVF file for VSAN 6.1 Witness was different or it is a newer version of VMware Fusion that can't parse the OVF file, but here is what I did to make it work. 

Just a small note - all credits for the following procedure go to William Lam. I only adjusted a couple of steps in his procedure to make it work for VSAN 6.2 Virtual Witness Appliance running in VMware Fusion 8.1.

1. Extract content of the OVA file using VMware OVF tool.

/Applications/VMware\ OVF\ Tool/ovftool VMware-VirtualSAN-Witness-6.0.0.update02-3620759.ova VMware-VirtualSAN-Witness-6.0.0.update02-3620759.ovf

2. Create a copy of OVF file in case something goes wrong. It is a quicker option than extracting OVA content again

3. Open OVF (you can even use default textEdit utility) and adjust the appliance size by moving the text marked in Red

<Configuration off:default="true" off:id="normal">

to the following string

<Configuration off:default="true" off:id="tiny">

3. Go to line 821 and delete the following text - marked in the screenshot

Btw, even though textEdit doesn't show line numbers you can still press Cmd+L and type the line number you want to go to. 

You can save and close OVF file now.

4. Create new SHA1 checksum for updated OVF file

openssl sha1 VMware-VirtualSAN-Witness-6.0.0.update02-3620759.ovf

5. Update the OVF file checksum in manifest file

6. Now you can import VSAN 6.2 Witness, but don't press Finish yet.

7. Get to the VM's locations and open the package to get to the .VMX file

8. Open the .VMX file - again, textEdit works just fine, add the following code and replace the password

guestinfo.ovfEnv = "<?xml version='1.0' encoding='UTF-8'?><Environment xmlns='http://schemas.dmtf.org/ovf/environment/1' xmlns:oe='http://schemas.dmtf.org/ovf/environment/1'><PropertySection><Property oe:key='vsan.witness.root.passwd' oe:value='Password123'/></PropertySection></Environment>"

9. Once you save the file you can run the Witness Appliance.

The new VSAN 6.2 has arrived!

The biggest release since VSAN 1.0 has been just announced.
VSAN gets mature and recently proved to be the leader of the HCI market.

Here is the list of new features that comes with VSAN 6.2:

• Deduplication and Compression 
• Erasure coding 
• Quality of Service (QoS) using IOPS limits 
• New performance service and improved health service 

Let's do a quick review of each new feature:

Disk space efficiency

Deduplication and Compression features are presented in VSAN 6.2 to reduce amount of consumed storage by as much as 7x. This is best-case scenario and average ratio, I guess, will be around 2.5-3X.

Both technologies are available in All-Flash VSAN configuration only and are available as a single feature. You can't enable Deduplication without enable Compression and vice versa.

By default, Deduplication and Compression are disabled. To enable it ESXi servers will need to reformat all disk groups which can be pretty lengthy process, depending on the VSAN size. But don't worry - this procces doesn't require any downtime of your VMs. In worst case scenario some redundancy reduction will be incurred, e.g. like in the screenshot below

First, the ESXi server will dedupe data during de-staging phase when data is copied from the cache tier to capacity tier. The VSAN uses 4K blocks of data when looking for identical blocks. The data is deduped only within a single disk group.

Once data is destined and deduplicated ESXi host will try to compress 4K blocks of data, but it will only do so if the compressed block of data is less than 2K, otherwise the data will be left uncompressed

Note - The same rule is applied to compression of memory pages during the RAM contention. I am wondering if VMware simply re-used the compression algorithm. :)


RAID-5/6 which goes under name Erasure Encoding

That's the feature a lot of VSAN fans have been waiting for.

Prior to VSAN 6.2 the only option to provide data redundancy was mirroring which is quite expensive. RAID-5/6 , on the contrary is significatnly cheaper, but comes at performance cost, particularly, due to write overheads when parity data has to be recalculated every time any block of the stripe is altered. Since Erasure Coding is available in All-Flash configuration only the performance impact can be safely neglected for most of the environments.

FT=1 setting will use RAID5 and FT=2 will implement RAID6.

Another new feature introduced in VSAN 6.2 is Swap Efficiency. With this setting enabled all Virtual Machnie swap files will be created as sparse object. That is, the disk space won't be actually used for swap files until memory contention happens and ESXi host starts to swap out RAM pages into swap files. Along with Deduplication and Compression this swap efficiency technology provides very significant disk space savings.



Quality of Service (QoS) using IOPS limits

Now VSAN gets closer to traditional VMFS datastores in terms of storage QoS. The first releases of VSAN didn't allow IOPS limits/shares for VMs residing on VSAN datastores since the SIOC wasn't compatible with VSAN. With VSAN 6.2 administrators will have an option to limit IOPS per VM to be able to control "noisy neighbours".



QoS for VSAN 6.2 doesn't make any distinction between Reads and Writes.


It ought to be noticed that all IOs are normalised to 32KB block size. That means that if you, for instance, set a IOPS limit to 100 and block size of the IOs will be up to 32KB that VM will be able to get 100 IOPS. However, with block size of 64KB the VM will get only 50 IOPS.


New performance service and improved health service

VSAN 6.2 delivers new graphs and data to provide performance information at the cluster, host, virtual machine and virtual disk levels.

Interestingly, that performance data isn't stored in vCenter database. It is stored in as a separate VSAN object and has no dependecy on the vCenter.

Here are some examples of new types of graphs.


For instance, on the cluster level you can compare Virtual Machine IOPS and Throughput stats with Virtual SAN Backend stats. The latter shows the 'raw' amount of IOPS produced to deliver Virtual Machine IOPS, e.g. with Failures to tolerate set to 1 for each VM Write IO the VSAN backend has to produce 2 Write IOs to mirror data to two different hosts.


On the host level you can see Disk Group and Disk detailed stats in addition to VM IOPS and Throughput graphs mentioned above. So you can see how well-balanced your disks groups if you have more than one per host.



You can also see the analysis of used capacity now.

Health Services now has even more items to test and monitor. It is enabled by default and runs all tests every hour.

Apart from checking the health of all key services and different aspects of VSAN configuration (Networking, Storage, etc) it also checks all physical components of VSAN, e.g. controller driver, against VSAN Hardware Compatibility List.


Finally, VSAN 6.2 makes it way easier to upload support bundles to VMware Support to speed up the troubleshooting process.

Even though we haven't seen any new features of vSphere there is huge amount of information to digest - just have a look at other announcements of VMware Workspace One and Horizon 7.