Nested VMware 64bit VMs. Time to run Hyper-v or KVM inside VMware ESXi5.0 and Workstation 8.0

Working with multiple hypervisors has been great in experimenting with certain features or just gaining an understanding how to configure something similar to a customers environment. Until now my lab environment is littered with multiple servers  or harddisks with different images upon them but having the ability to work  with these technologies with minimal hardware in lab environment or even mobile out and about would be great. Until recently this was not possible due to the fact that nested 64bit VMs were not possible with Workstation 7.x or even ESX 4.x for that matter. But now with ESX5i and Workstation 8 this is now possible.

One caveat to add though is that your hardware must be VT-x capable which of course most are these days but with one additional and really important CPU instruction set of EPT (Enhanced Page Tables) or RVI from AMD must be present, searching the various chip manufactures sites it seems that if your running Nehalem or i7 from Intel at least then your good to go. Luckily I have both an i7 on my laptop and Nehalem in my HP ML G6 home test lab. So lets take a look at ESX5i first and see what needs to be set for Hyper-v to load.

VMware ESX5i configuration.

1. First you need to enable hardware virtualization by modifying the etc/vmware/config file. Enable SSH via tech support mode and putty to the ESX5i server

2. Once connected with putty  :

# echo ‘vhv.allow = “TRUE” ‘ >> /etc/vmware/config

3. Next create your Virtual Machine hardware, I personally used hardware version 8 to make things easier with configuration.

4. Before you get to booting up the VM and installing Hyper-V you need to add two lines the virtual machines config file .vmx

You can either add these via the vSphere Client in the settings of the virtual machine > Configuration Parameters, or doing it from command-line

To add them using command-line move back in SSH > change into the directory where you Hyper-V VM is installed

For example config file where my VM is located is called Hyper-V.vmx. Type the following commands:

# echo ‘monitor.virtual_exec = “hardware” ‘ >> Hyper-V.vmx

# echo ‘hypervisor.cpuid.v0 = “FALSE” ‘ >> Hyper-V.vmx

5. Next there are a couple of changes to be made with the CPU configuration.

in the VM settings > Options > CPU/MMU
make sure you select the option to pass the Intel EPT feature.

6. Next move to the Options area > CPUID Mask click on Advanced


Add the following CPU mask Level ECX: —- —- —- —- —- —- –H- —-

8. Finally you are now ready to install Windows  2008/R2 and enable the Hyper-V role.

VMware Workstation 8 configuration.

For configuration with Workstation 8.0 things are a little easier.

1. Start off by creating a new VM again I used version 8 hardware

2. Configure the VM with RAM and Disk etc….

3. Important step here select Windows 2008 R2 x64  and not ESX5i as you probably did with nested 32bit VMs.

4 Under the settings of the VM > CPU, make sure  you have the option to pass-through the Intel VT-x/EPT feature.

6. Make sure you have set the VM to boot from  Windows 2008 R2 x64 media ISO.

7. Before booting, you should edit the config file  .vmx and add the parameter: hypervisor.cpuid.v0 = “FALSE”

8. Now Boot and Install Windows 2008 R2 x64.

Next stop for me is to see if I can get KVM working also and then I will be happy.

5 Replies to “Nested VMware 64bit VMs. Time to run Hyper-v or KVM inside VMware ESXi5.0 and Workstation 8.0”

  1. Hi Lorenzo,

    Thank you for this excellent reference. I’m attempting to create 2nd level VMs in KVM on top of ESX5. I know of someone who has done this successfully on Workstation 8, but I am having issues. The 2nd level KVM VM hangs on boot and the qemu proc on the 1st level VM spins it’s CPU at 90+%. Appreciate if you have any insight.

  2. Hi,
    We are actually, trying to work with nested VMs and KVM. Our model is, VMWare Workstation and a Ubuntu Linux version on it as VM and our KVM hypervisor of type 2 on Ubuntu and Archlinux on this KVM. so our concern, is how efficient will our system utilize and work, with respect to hardware virtualization capabilities of the main host system. How can we have a measure of this?

Leave a Reply

Your email address will not be published.