Installation of virtual openSSI cluster on Qemu
From OpenSSI
Qemu OpenSSI virtual cluster howto
Author: Mulyadi Santosa (mulyadi.santosa@gmail.com)
Feel free to correct the information in this document. Please kindly send me your diff against this text file in case you found errors. I do appreciate your effort on reviewing and testing the steps below on your own Linux system.
This document explains the general steps to create virtual cluster using qemu 0.7.0, openSSI 1.2.0 and Redhat 9 Shrike
1. Create disk image with "qemu-img" $ qemu-img create redhat.img 1500M --> adjust the size according to your need
2. Create a script file containing these lines
#!/bin/sh
sudo /sbin/ifconfig $1 0.0.0.0 netmask 255.255.255.0
save this as "qemu-ifup" on the same directory where you create the above image.
3. boot and install Redhat (install Redhat just the way you do normal installation)
This step is necessary to create first node.
$ qemu -cdrom /path/to/mount directory of redhat-CD -hda redhat.img \
-boot d -m 256 -n qemu-ifup
Note: Use the above command everytime you need do something inside Qemu VM (Virtual
Machine).
Warning:
- Sometimes keypress is somehow ignored, consult qemu-devel mailing list
if this happens (I think this is an SDL bug)
- Personally, I create swap size with size smaller than 256MB. Does anyone
successfully create swap bigger than that?
To conserve space, I suggest to install only the following categories
Administration Tool
Editor
Development Tools
kernel development
They take about 950 MB
NOTE: Choose GRUB over LILO to ease instalation of openSSI packages.
USING LOMOUNT !! Pay attention for a moment!! :
There is a big chance that after step #3 above, you need to transfer something
inside the disk image. To ease your day, use "lomount" utility from
http://www.dad-answers.com/qemu/utilities/QEMU-HD-Mounter/lomount/ (just compile
the C file).
Using "lomount", you can do loopback-mount the disk image e.g
# lomount -t ext3 -diskimage redhat.img -partition 1 /mnt/mounted
-t : partition type of target partition inside disk image you want to mount
-partition : partition number, 1 for first partition, 2 for 2nd, and so on
last parameter is mount point.
With the target partition being mounted, feel free to copy files from/to disk image.
WARNING: DO THIS WHEN QEMU IS NOT RUNNING !! YOU WILL CORRUPT THE DISK IMAGE IF YOU DO THIS WHILE QEMU IS USING THE DISK IMAGE !!
4. Do glibc update. On the past, i do these on both host and inside Qemu. I wasn't sure whether we need to do it on host. To update glibc inside the Qemu virtual machine, use lomount. a. Use openssi-rh9-1.2.0-glibc.i686.tar from openSSI download area and follow
the instructions.
b. You need to fix the "RPM" mess then. refer to "RPM fix" written by me
on openSSI wiki. DO THIS AS SOON AS YOU HAVE DONE STEP #1 !! In short, do:
1. backup the files on /var/lib/rpm and then delete them
# rm /var/lib/rpm/__db*
2. rebuild database RPM
# rpm -vv --rebuilddb
In case something wrong happen, just revert back to the back-up copy
of RPM databases (step b.1)
5. After successfully install redhat 9, now do kernel compilation. We need to do it to avoid boot problem. I prefer to do compilation on host (not inside the Qemu) to do things faster.
c. Install kernel-ssi-source-2.4.22-1.2199.nptl_ssi_6.i686.rpm and change
the working directory to /usr/src/linux-2.4.22-1.2199.nptl_ssi_6.
d. Do kernel config.
!! Use!! CONFIG_HIGHMEM4G=y !!instead of!! CONFIG_HIGHMEM64G=y !!
You can disable SMP feature because Qemu doesn't support SMP mode.
e. Set the following options for network card support
CONFIG_NE2000=y
CONFIG_NE2K_PCI=y
f. Disable Direct Rendering Manager (DRM) because it's not needed
# CONFIG_DRM is not set
g. Configure IP virtual server sebagai native kernel
CONFIG_IP_VS=y
Note: step f and g are neccessary to avoid "Unresolved symbol" warning during
"make modules_install".
Do the usual kernel compilation steps including modules installation.
After compilation is all finished, copy the kernel image (including the
System.map and vmlinux if you wish) and kernel modules binary
(/lib/modules/<SSI-kernel-version>) inside the disk image.
CREATE directory with the same name as the original one which hold
openSSI kernel modules (e.g /lib/modules/2.4.22-1.2199.nptl_ssi)
6. Copy openssi-rh9-1.2.0.i686.tar to the Qemu's disk image. Run the Qemu VM and do normal openSSI installation. Read "Installing an OpenSSI 1.2.2 Cluster on Red Hat 9" on http://www.openssi.org/cgi-bin/view?page=docs.
When installation is done, DON'T get rush and doing reboot !! Set the "default=" parameter to point back to the original (non SSI) kernel image so you don't accidentally load the "buggy" kernel image which can cause hang when kernel is loading.
7. Create initrd image against YOUR SELF-COMPILED kernel modules, e.g:
#mkinitrd --cfs /boot/openssi/coba-initrd.img 2.4.22-ssicustom The first parameter is the name of the initrd you want to create. The 2nd parameter is the directory which contain YOUR SELF-COMPILED kernel modules. Add comment mark "#" in front of related lines in /etc/modules.conf which sets eth0 alias so mkinitrd can work properly! (Why? because we compiled them as part of kernel, not as module).
8. Add entry inside grub.conf to load your self-compiled kernel image along with the initrd image you just created on step #7. Adjust the "default=" parameter to pioint to the new kernel image/initrd . Try to use the same kernel parameter like the "original"openSSI kernel use.
Personally, I prefer to manually execute "ssi-ksync-network" to prepare the kernel and initrd image for tftp boot.
9. Now boot you new openSSI kernel and make sure the overall openSSI functionality
works properly.
10. Read the qemu-vde-howto 0.3.1 (http://lists.gnu.org/archive/html/qemu-devel/2005-06/msg00316.html) about setting up VDE interface. Of course, you need to compile and install related package first. This is a method I choose to simplify the creation of virtual network.
Quick steps to do it:
a. #!/bin/bash
vde_switch -hub -tap tun0 -daemon
/sbin/ifconfig tun0 192.168.1.13 netmask 255.255.255.0
Save it as "vde-start.sh". Use it to initialize network interface.
It is important to use different network id from the one used by host,
for example I use 192.168.1.x because on my PC I have an ethernet card
with IP 10.1.1.1 and netmask 255.255.255.0
b. #!/bin/bash
/sbin/ifconfig tun0 down
pkill vde_switch
Save it as "vde-stop.sh". Use it to terminate network interface and
to stop vde_switch.
c. Now, run Qemu VM using this "improved" command
#!/bin/bash
vdeq qemu -hda redhat.img -m 256 -boot c
To save time, save it as "qemu-start.sh" or anything you like.
d. Inside the Qemu VM, set the IP address and netmask of the virtual
"eth0" to match the VDE interface's netmask e.g 192.168.1.11/255.255.255.0
Set the default gateway to point to VDE interface IP (this is especially
needed if you want to communicate with host e.g doing rcp)
Keep this "init node" running and now prepare the 2nd node. Try to ping
the Qemu guest system from your host system to ensure network communication
is working properly!
11. Now, for adding the 2nd node and so on, download Etherboot package (http://etherboot.sourceforge.net) and compile floppy disk image for ne2000 (ne.dsk). Use this to boot second node:
# vdeq qemu -fda ne.dsk -isa -m 256 -boot a -macaddr 52:54:00:12:24:59
Here I use explicit "-macaddr" option to avoid conflict with the MAC addres of the first Qemu VM (init node). You can use any any number as long as it is unique. -isa is used becaused this is the only way I know to make Etherboot recognized the virtual eth0 on Qemu VM (ISA emulation instead of PCI). You may try ns8390.dsk instead. See here: http://lists.gnu.org/archive/html/qemu-devel/2004-12/msg00000.html
12. Follow the described steps on openSSI documents to add the additional nodes. Switch screen focus between init node's display and slave's display (Qemu VM window) to manage them at once. Remember to set use the same netmask as the first node has.
Happy fun the virtual cluster !
reference
A. qemu-vde-howto 0.3.1 http://lists.gnu.org/archive/html/qemu-devel/2005-06/msg00316.html
B. Installing an OpenSSI 1.2.2 Cluster on Red Hat 9 http://www.openssi.org/cgi-bin/view?page=docs
C. lomount utility http://www.dad-answers.com/qemu/utilities/QEMU-HD-Mounter/lomount/
