Multibooting Linux, Windows, and Mac OS X on a Mac and using them as Virtual Machines

Recently, I set up my Macbook to boot OS X, Ubuntu, and Windows XP, with a large common partition that they all can natively read and write.  Then I decided it’d be nice to be able to interact with these other systems through OS X in VMware Fusion, so I set that up as well.  This took many tries, and a lot of experimentation.  In the spirit of science, after I had everything working, I documented it, and then started from scratch to verify my instructions.  I wrote up my instructions along with a dabble of theory.  They follow.

This is what shows up when I boot.

If you try this, make sure to have a backup of everything.

To multiboot a Mac with OS X 10.5.5, Ubuntu 8.10, and Windows XP SP3 circa November 2008:

Ingredients:

• Intel Macintosh
• OS X 10.5 Media
• Ubuntu 8.10
• Windows XP SP2 or higher media
• VMWare Fusion (If you want to be able to run Windows and/or Linux inside of OS X.)

There are many ways to get this working, but also many ways that do not.  One way that works is this:

1. Boot the Mac from the OS X install media.  Instead of installing right away, go Disk Utility.  Use Disk Utility to partition the hard drive into four partitions, Linux, Share, Windows, and Mac.  Set the first three to be MS-DOS (FAT).  Set the Mac partition to be Mac OS Extended (Journaled).  Apply this.  Quit Disk Utility.
2. Install OS X on the Mac partition.
3. Reboot and set up your administrator user.  Do all software updates.  Reboot.  Check for software updates again.  Reboot and repeat until no updates are available.
4. Install rEFIt.  rEFIt is an EFI boot menu.
5. Set the windows partition to be active (bootable).  This can be done in OS X through the terminal.  Open the terminal (/Applications/Utilities/Terminal.app) and run the command: sudo fdisk -e /dev/rdisk0
6. You will likely get the error, “fdisk: could not open MBR file /usr/standalone/i386/boot0: No such file or directory”.  That’s ok.  Type p to print. You should see four partitions.  If you look carefully, the first partition is a 200 megabyte one and what you thought was partition one is actually partition two.  OS X put an EFI System partition in the first spot when you formatted the disk, so you actually have five partitions on the disk.  Your last partition, the Mac one, shouldn’t be visible from this screen.  The disk is a GPT disk, which contains inside of it a Master Boot Record (MBR). This MBR is how older OSes find partitions and bootloaders.  With Apple, this MBR doesn’t support extended partitions, so you can only have your four primary partitions.  That’s what you’re seeing here.  You have to type f 4 to set the Windows partition as bootable.  After that, press q to quit and then y to save changes.
7. Insert the Windows XP media and reboot.  Select the Windows CD when rEFIt asks how it should boot.
8. After loading drivers, Windows will ask where it should install.  Once again, it’s going to see four partitions.  The 200 meg EFI partition, and the first three partitions you defined.  I’ve tried this step in many, many ways, and the only times I’ve been successful I’ve done the following:  Make sure that the install partition is labeled C: and is the last partition shown.
9. Select it, and format the partition.  A FAT32 quick format and long format both work.  I haven’t tested NTFS.  If I didn’t format the partition, after Windows reboots and tries to continue the install, I have gotten “Missing Operating System” errors, “hal.dll not found” errors, as well as any other excuse Windows can find for why it cannot boot.  If a different partition is labeled C: than the Windows partition that we created at the beginning, quit the install, reboot into OS X, and go back to step 5.  See if there is a little asterisk next the partition we’re trying to install to.  That means active and bootable.  From my experiments, it looks like Windows sets C: to the the active partition if there is one, otherwise it is first FAT/NTFS partition.  Why don’t we make this easy and install Windows to the first FAT/NTFS partition?  If Windows isn’t installed on the last partition in the MBR, it fails to install properly.  I’ve tried mucking around in boot.ini and other files with no luck. Windows will need to reboot twice while installing.  Finish the install.
10. Insert your OSX 10.5 disk.  This has all the drivers you need for your system.  If you don’t have it, or you have a developer edition that doesn’t have those drivers built in, you can extract the drivers from the Boot Camp Assistant application or find them on the internet.  Go through the “Boot Camp” driver auto-installer that autoruns, and then reboot.  When rEFIt shows up, select the Windows install on the hard drive, and do all the Windows updates. Reboot and continue to do updates until there are no further updates.
11. At rEFIt, select the partitioning tool to sync the GPT and MBR.  If it doesn’t need syncing, it will tell you.  Then select OS X and verify it boots properly.  Insert your Ubuntu media and reboot into the disk using rEFIt.
12. Install Ubuntu as you normally would, with two exceptions: partitioning, and the boot loader.  At partitioning, select the manual partition option.  It will scan your disks, and show you yet another view of the drive.  Check the format box on the Linux partition, and set it to mount at /.  This will be sda2 if you’re following the partitioning layout.  Then proceed.  The installer will complain about a lack of swap space.  If you need swap space, you can set it up as a swapfile after the installer.  Acknowledge the complaint and move on. At the last step before you let the install finish, there is a little button labeled “Advanced”.  Clicking that button opens a window that includes a dropdown box for setting the location where the GRUB bootloader is installed.  If you leave this as the default, you’ll have to go through GRUB for both Windows and Linux, which you probably don’t want.  Instead, set this to be the Linux partition (/dev/sda2).  You won’t have to tell Windows about this bootloader because rEFIt will use it directly.  After clicking next on this screen, Ubuntu will install and prompt you to remove the media and reboot the machine.
13. Verify Ubuntu, Windows, and OS X all boot by choosing them at the rEFIt menu.  Boot into OS X last, and remember which non-OS X OS you booted into last before choosing OS X. (You shouldn’t need to, but if you have issues see if you need to sync your partition tables between GPT and the MBR using the partitioning tool in rEFIt.)

At this point, if you don’t want to be able to use those installs in VMWare while booted into OS X, you’re done!  Otherwise, trek on, my faithful reader.

In OS X, install VMWare Fusion.  There’s nothing special about VMWare Fusion over any of the other products, its just the one I already have.  Start VMWare Fusion, and ignore the “Boot Camp Partition” entry.  There’s a bug filed at VMWare to allow people to remove it.  There’s not a real reason why you need to ignore it, but I’m going to teach you how to do it manually. You’re going to have to do it manually at least once to get Linux working, and its pretty easy.

To create a VM that points to a partition on the disk, so we can use a partition to boot into as well as run in a VM, we need to do the following.

1. Use vmware-rawdiskcreator to create a disk file that points to the partition.
2. Create a new VM, and remove any autocreated harddrives and add our raw disk file.
3. Edit the vmx preferences file to disable snapshots and suspend.  With these enabled, its too easy to corrupt your partition.

vmware-rawdiskcreator is a command-line program that allows VMware VMs to directly address partitions instead of using a file as a hard drive.  It can only address the MBR, so there are the same restrictions on which partitions can be pointed to as with Windows.  At this point, I’m going to copy and paste the help.

tredegarh:~ wolf$ /Library/Application Support/VMware Fusion/vmware-rawdiskCreator –help
Usage:

/Library/Application Support/VMware Fusion/vmware-rawdiskCreator print <diskDev>
Print the partition table of a physical disk.

/Library/Application Support/VMware Fusion/vmware-rawdiskCreator hasBootCamp <diskDev>
Check if a physical disk has an Apple Boot Camp partition (exit
code is 0) or not (exit code is non-0).

/Library/Application Support/VMware Fusion/vmware-rawdiskCreator create <diskDev> <partNum> <virtDiskPath> <adapterType>
Create a VMware virtual disk backed by a partition of a physical
disk.

<diskDev> is a disk device, e.g. “/dev/disk0”.

<partNum> is the partition number as printed by the “print”
command, e.g. 3. 0 is special and means “the Apple Boot Camp
partition”.

<virtDiskPath> is the path of the virtual disk description file to
create, e.g. “~/Virtual Machines/My VM/My Raw Disk”. Two files
will be created: “<virtDiskPath>.vmdk” and
“<virtDiskPath>-pt.vmdk”.

<adapterType> is the virtual disk type. It must be one of “ide”,
“buslogic”, or “lsilogic”.

<virtDiskPath>.vmdk is a human-readable parameters file.  It identifies the vmdk as a raw disk and points to the device representing the partition.  When we’re done with this, the booted VM is going to see this raw partition as the only partition on a virtual device.  What’s going to happen when the VM wants to read the MBR and partition tables of the virtual hard drive?  Those aren’t stored in the partition we’re pointing to.  This is where the second file comes in–the <virtDiskPath>-pt.vmdk.  According to my experiments, this file is a copy of the drive’s (<diskDev>) MBR and some modified partition tables.  This is created when vmware-rawdiskCreator is run.  It isn’t dynamic.  This is why we have to remember which non-OS X partition we booted last.  rEFIt marks a partition active/bootable when we select it from rEFIt’s menu.  If we make a raw disk file pointing to a non-bootable partition, the bootloader won’t start right when loaded by the virtual machine.

Back to the doing!

Assuming we’re following my partitioning layout, and assuming we booted Linux last, we run
/Library/Application Support/VMware Fusion/vmware-rawdiskCreator create /dev/disk0 2 linux ide

Now we need to create a VM.  Open VMware Fusion, and create a new VM.  When it asks about install media, skip with “Continue without disk,” then “Create a custom virtual machine.”  Pick the closest OS you can, then finish the wizard.  Then VMware opens up the System Settings.  In the Hard Disks section, remove the automatically created hard disk.  If you want, remove the automatically-created vmdk files inside your VM.  You’ll have to close the virtual machine from inside of VMware in order to remove some lock files.  To remove the rest of the files, you can use the Finder or the Terminal or however else you want to remove files.  Virtual Machines are just directories but they look special to the Finder.  The VM will appear as a bundle (just like applications) probably inside of your Documents/Virtual Machines directory.  You can right-click on the virtual machine icon inside of the Finder and “Show Package Contents” to get to the contents of the virtual machine, or else you can just use the Terminal.  Everything ending with .vmdk inside of this new virtual machine directory is safe to remove at this point.

This is a good time to disable snapshots and suspending from this VM.  To do this, edit the .vmx file in the first level of the VM bundle.  Add the lines:

suspend.disabled = “TRUE”
snapshot.disabled = “TRUE”

and save the file.

Open VMware Fusion again, and get to the Hard Disks section of System Settings of your new virtual machine.  Add a new disk.  Choose an existing disk, and navigate to the place where you saved the .vmdk and -pt.vmdk file.  Select the .vmdk file.  Copy it or move it, depending on which you’d rather do.  Apply, and then start the virtual machine.

A common error at this point is “Missing Operating System” or VMware Fusion complains that there are no bootable devices.  This can be a problem with the raw disk files.  To fix this, remove the hard drive, rerun vmware-rawdiskcreator after rebooting into the OS through rEFIt and rebooting back into OS X in order to get the boot records correct.

At this point you can install VMware Tools.

To set up the VM for the other OS, reboot into it through rEFIt, and then back into OS X.  You could also use fdisk -e like we did earlier, but this way lets you verify that the other OSes are still bootable.  Once you’re back in VMware, create the disk for the other OS through rawdisk-creator and follow the same process we did for the first OS.

Now, you might notice an issue with the share partition.  The share isn’t accessible from more than one OS at the same time.  This is for filesystem consistency reasons.  If more than one OS were editing the drive at the same time, corruption would likely occur.

How do we fix this?

We can use file sharing through something like Samba or NFS, or Shared Folders through VMware for the VMs.  Windows and Ubuntu will still be able to access the share when they’re booted natively, and through Shared Folders when they’re in a VM.

At this point, you should have a triple-boot Mac OS X / Windows / Ubuntu Linux system.   From inside OS X, you can open the Linux and Windows partitions as VMs.  Each OS can see the share partition when natively booted, and the virtual machines can see the shares if you set up Shared Folders.

Why did I choose this partitioning layout?

It was a lot like a logic puzzle, with little facts teased out after repeated installations.

• Windows needs to be in the last spot in the MBR or you get install issues on an Intel Mac with OS X.
• The EFI System partition is rumored to be used in firmware updates, so I left that in the first spot, where it puts itself.
• Grub needs to be installed on a partition located in the MBR, but Linux doesn’t.
• Windows can only see partitions in the MBR.
• OS X can see all the partitions.
• Linux can see all the partitions.
• I want a share partition that can be read by all OSes without installing programs.  This means FAT32.  To be seen by all OSes it needs to be in the MBR.
• OS X doesn’t need to be installed in the MBR.

Let me know how it works out for you guys.