Recently, I had the need/desire to PXE boot one of my machines into VMWare ESXi 6. Not to do the install via PXE boot, but to boot into an already installed, fully functioning ESXi environment.
Lacking any clear tutorial online, I had to do some trial and error. This is my story.
Aside from a working PXE server, the first thing you need is to have already installed ESXi somewhere. For me, that was a USB key. ESXi can be installed in less than 1GB of space, so I chose a simple 1GB USB key I had laying around and did the install.
Your next step is to extract the data from the USB key. dd on Linux or BSD is perfect for this:
> dd if=/dev/sdb of=image.img
Once you have that image.img file, you’ll have to extract the data off of it. But wait! ESXi installs into multiple partitions, so what you have right now is an image file with 5 partitions on it. (There are plenty of places out there that talk about the partition layout, such as this one here, so I won’t bore you with the details).
We’re interested in the data from partition 1 and partition 5, specifically. Partition 1 contains the boot files we’ll need, and partition 5 contains the actual ESXi installation.
NOTE: We'll actually be disregarding a few items here that other folks might find key, namely the crash dump partition, the second 250MB partition for in-place upgrades, the VMWare Tools ISO partition, and any scratch partitions (assuming your USB key was larger than 4GB). If you need those, well...I can't help you. You'll need to do more investigation to get those running.
The easiest way to extract the data from those two partitions is by mounting them up. I use kpartx for this:
> sudo kpartx -av image.img add map loop0p1 (253:0): 0 8128 linear /dev/loop0 64 add map loop0p5 (253:1): 0 511968 linear /dev/loop0 8224 add map loop0p6 (253:2): 0 511968 linear /dev/loop0 520224 add map loop0p7 (253:3): 0 225248 linear /dev/loop0 1032224 add map loop0p8 (253:4): 0 585696 linear /dev/loop0 1257504 > sudo mount /dev/mapper/loop0p1 /mnt/temp
We can then change directory to /mnt/temp, and copy the data off.
From the first partition, we simply need mboot.c32. The rest can be left behind.
From the second partition, we need everything, so copy it off to your PXE server’s TFTPRoot (or a subfolder therein).
When you’re done, unmount the device, and use kpartx to tidy up:
> sudo kpartx -dv image.img del devmap : loop0p8 del devmap : loop0p7 del devmap : loop0p6 del devmap : loop0p5 del devmap : loop0p1 loop deleted : /dev/loop0
Over on your PXE server, we need to put the mboot.c32 file from the first partition, and boot.cfg from the fifth partition into the same folder. That can be the root of your TFTP server, or a subfolder; it’s up to you. I chose the root of my TFTP server.
Next, we’ll modify the boot.cfg file slightly, by adding a prefix line. NOTE: if you put your mboot.c32 into the same folder as the contents of partition 5, you do not need to add a prefix line. The prefix line is there to tell the system that the boot files are in a location other than where mboot.c32 and boot.cfg are located. Thus, this can be avoided if mboot.c32 and boot.cfg are placed into the same folder.
bootstate=0 prefix=/esxi/ kernel=tboot.b00 title=Loading VMware ESXi
The format is prefix=/partition5folder/, where “partition5folder” is the folder that you copied the contents of partition 5 to. Leave the rest of that file alone.
After that, we need to modify our default file in the pxelinux.cfg folder. For that, you simply need:
Note that if you put mboot.c32 and boot.cfg into somewhere other than your TFTProot folder, you’ll need to provide the path here.
Save the default file, and you should be booting into ESXi in no time!