Wouldn't it be nice ...

I'll be putting together a small, diskless, mixed-node Gentoo cluster this week and hopefully detailing my progress on here. I recently got some interesting ideas from a fellow dev, Christian Zoffoli, on the gentoo-cluster mailing list. He had a setup something like this:

What it should do:

• Power on.


• Start PXE -- look for DHCP, TFTP servers.


• Download the kernel and a uclibc initrd.


• Mount an NFS shared image (one per cluster).


• Autoconfigure.


• Start a full Gentoo system.

Technical details:

• Create a diskless boot setup. Set up DHCP, TFTP and NFS servers. The DHCP server tells nodes their hostnames and where to go next to get a kernel. (Remember to use -H to dhcpcd on the client or else it ignores the hostname.) The TFTP server provides the kernel and initrd, and the NFS server provides a full root filesystem.


• Create a layer using an embedded environment so we can have a diskless node up and running quickly. ROOT support in Gentoo's Portage should allow us to do this fairly easily.


• The linuxrc within the initrd does much of the real work.


• Mount /proc.


• Read the hostname from /proc/sys/kernel/hostname.


• Parse /proc/net/pnp to find the boot server.


• Download a more complex linuxrc (linuxrc.real) from a TFTP server, so there can be a simple, statically generated initrd.


• Execute linuxrc.real


• Mount the NFS root with a full Gentoo installation in a common path [COM_P1].


• Mount a tmpfs in another common path within the prior one [COM_P2].


• Generate resolv.conf, hostname, dnsdomainname, etc. in $COM_P2.


• Download SSH keys (common to all nodes) to $COM_P2.


• Mount a tmpfs in /tmp, /var/lock, /var/run, /var/lib/init.d and /var/log. The logger sends messages to a primary logger on the master.


• Link $COM_P2 to the real location (in $COM_P1).


• Link and download wathever you need.


• Switch roots to the NFS share using pivot_root.


• Execute /sbin/init in $COM_P1 to start the normal Gentoo initialization.

Corey, what type of graphics would you prefer? The Radeon 9000 is pretty sweet for a laptop. You've got two different sets of drivers to use -- the open-source ones from the DRI project and fglrx, straight from ATi.

Matthias, never, ever do anything like this again.

Seemant, if you think /var/lib/heimdal is more appropriate, why don't you try to push it upstream instead of just deciding for us? If it's a good idea, they'll probably like it too.

Quick poll: If you're on a planet or read a planet, which one is it and how often does it update?

I'm beginning to wonder whether sending the hostname via DHCP makes any sense. We'll still need DNS for name-based internode communication so we don't need to deal with static /etc/hosts on every node or some other substandard solution. Maybe I'll just forward the nodes to a DNS server and deal with hostname from there.