performance tuning vmware

There are many pages of information on the Internet about performance tuning ESX and I don't want to rehash what's been said before. VMWare themselves have an excellent summary here vi Performance Tuning

However... if you've found yourself at this little page you're probably scratching your head trying to work out why your system isn't working as well as you expected.

Ok, so I'll rehash a little. Maybe state the obvious a bit.

Virtualization isn't always the answer
Virtualization is useful in a lot of cases, but there some times where physical is better. Lets face it, if you could afford to run your app on 1000 real servers you would yeah? But you're using Virtualization to save money I presume. For some workloads it makes more sense to use physical servers. If you need as much CPU grunt as you can get, or your workload is memory hungry you may be better off physicalizing some parts of your system.


Benchmark, benchmark, benchmark
You need to know what kind of performance each tier of your system needs to have and actually has. The other day I was horrified when I moved an application from a physical server to a virtual machine. The physical server was able to spit out pages of HTML at a rate of about 1000 per second. The VM was only able to serve 10 per second. I spent a while gnashing my teeth, then took a step back and worked out, my particular system only needed 10 pages per second. I'd never need this component to perform that fast. The objective is to maximize bang for buck not absolute performance. Know the performance characteristics of your whole system.

Benchmark every component of your system. If you run virtual database servers, know how long your SQL statements take. If you're running a website use some of the excellent tools around to benchmark your site. For some basic throughput tests try apachebench (ab). For more complex testing try OpenSTA or JMeter. If you have the cash the Mercury suite (LoadRunner) is the Rolls Royce of test tools.



Think Small
In the physical world you want as many CPUs as possible and as much memory as possible. In the virtual world this rule of thumb is turned on its head. Keep your VMs as small as possible and they'll run faster. (On a heavily loaded ESX server (or cluster) that is). Try to trim down the RAM on your VMs to the smallest value you can without having your VMs running into swap. If you're running a java stack keep your JVM (-Xmx) as small as you possibly can without causing the dreaded java.lang.OutOfMemory errors.

Spend some time trying out different combinations of servers and VM configs. Trial and error may lead to some surprising conclusions.

Identify your bottleneck
If you have a performance problem it's certain to be one of

• Network
• CPU
• Memory
• Disk
  

Which is it?

Check out your Virtual Infrastructure graphs.

Connect as many NICs as you can. Buy as many CPU cores as is economical. Use fast storage. RAID 10 preferably. If you're running database servers definitely use RAID 10.

Don't run too many Virtual Machines on a Physical Host
Sorry to state the bleeding obvious here, but it needs to be said. The problem is, it's too easy to create more and more VMs on your ESX infrastructure, until you get to the stage where the whole shebang grinds to a screaming halt.



So what's a good rule of thumb? How many VMs is too many?


If you're not maxing out CPU, network or disk take a good hard look at your RAM usage.

If you want maximum performance out of your infrastructure do not overcommit memory at all.
Don't believe what Virtual Center says.





Login to your ESX host with an ssh client.
Type esxtop
type m to show memory stats
Look top right.

Should say:
MEM overcommit avg: 0.00, 0.00, 0.00

If MEM overcommit is greater than 0 you may be in for performance problems as ESX uses tricks to squeeze too many VMs into RAM.