FreeNAS – Speeding up my NFS Datastore for XenServer
While using a nested XenServer 7.1 on VMware ESXi 6.5 to test something unrelated I noticed that running sdelete on one of my Gold Image VMs was painfully slow. Looking on the XenServer performance I could see that the write latency was nearly 100ms to the NFS SR which would explain it being painfully slow on a VM.
I’d already set up Jumbo Packets on my storage network for my Freenas server as well as both the ESXi host and the nested XenServer so couldn’t see that being a problem but decided to do a few ping tests in case.
Sending a ping 9000 in size from my ESXi host to the Freenas server worked fine but what surprised me was that it was too large from the XenServer. I dropped the MTU size on the XenServer to 8900 and pinging that size to the Freenas server worked fine.
That improved things very little so I decided to have a poke around the internet and found several articles about NFS being slow due to synchronous writes as default on Freenas.
Further digging came up with other specific articles for Freenas regarding disabling sync writes in ZFS also so I disabled sync writes for the Data Set holding my XenServer NFS SR. I then noticed a dramatic drop in write latency on the XenServer coupled with a much faster sdelete of the disk. I haven’t done any real testing as such as it’s just my trash and burn environment but just the responsiveness of the VM and sdelete speeding up considerably was a good indication to me that I’d found the culprit !
It was a good one to learn as I haven’t really done much with NFS for a long, long time and the virtualisation hosts are normally looked after by other teams on most customer sites I visit.
Next is to look at whether the same affects my iSCSI datastores too as you never know !