Dec. 4th, 2010

NFSROOT

Dec. 4th, 2010 05:53 pm
jimx0r: (Default)
So, I'm building a cluster for another department on Campus...

They're CentOS fans, so we're using CentOS (not my first choice, but... ). Anyway, we're doing a "diskless" configuration for this machine: While there /is/ a local disk on each node, the theory is that the diskless configuration will be a little simpler for them to manage. Its only my job to set it up for them, then give them a (all too brief) tour of the system and how to manage it.

The Diskless config is taken care of by the system-config-netboot tool under CentOS. It creates a somewhat interesting configuration. Basically, there is a shared NFSROOT mounted by all of the compute nodes, then, over the top of this is mounted a set of files/directories that should be unique to each machine. The mount table is a complete mess(!), but, to be perfectly fair, it seems to work pretty well. This simplifies the task of administering the cluster to basically maintaining two copies of the OS: one for the "head node" and one for the "compute nodes".

This got me to thinking...

If we're willing to have the mount table a bit, umm..., large, then why not have a single copy of the OS to maintain? Instead of a chroot environment that lives underneath the root filesystem somewhere, why not just NFS export / to all of the compute nodes, then do the same NFS mounting trick on the compute nodes in order to make them unique? Then we would have a single copy of the OS that is modified where, and only where, it needs to be unique for the compute nodes.

I'm thinking this kind of approach will likely represent a larger investment up-front, but I'd be willing to bet that your investment will be repaid many times over as opposed to this maintaining two distinct root filesystems (one for the head node, one for the compute nodes). I think it should especially help when it comes to preventing "version skew" between the software installed.

Anyway, that's my big idea for the day...
jimx0r: (Default)
So, I'm thinking a lot about booting over the network lately. I'm wondering about filenames for PXELINUX. PXELINUX will load its configuration via TFTP with any of these filenames:
/tftpboot/pxelinux.cfg/01-88-99-aa-bb-cc-dd
/tftpboot/pxelinux.cfg/C000025B
/tftpboot/pxelinux.cfg/C000025
/tftpboot/pxelinux.cfg/C00002
/tftpboot/pxelinux.cfg/C0000
/tftpboot/pxelinux.cfg/C000
/tftpboot/pxelinux.cfg/C00
/tftpboot/pxelinux.cfg/C0
/tftpboot/pxelinux.cfg/C
/tftpboot/pxelinux.cfg/default

So, what does that really mean to us??? Lets look at these entries individually:

/tftpboot/pxelinux.cfg/01-88-99-aa-bb-cc-dd
This first entry is the most highly preferred of the possible filenames you could give. It corresponds to a NIC with a MAC address of 01:88:99:aa:bb:cc:dd. This filename could be a good choice, but if you're using DHCP to hand out IP addresses to your machines that are netbooting, then you'll have to put that MAC address both into the DHCP daemon's configuration file and encode it into the boot filename here. Should the NIC die, you'll have to remember to change both this filename and the entry in the dhcpd.conf!

/tftpboot/pxelinux.cfg/C000025B
This entry, represents a host that has the IP address 192.0.2.91. This seems to me like a really reasonable choice to use for your boot filename. This way you're *not* having to specify the MAC in two places!

If you're wondering... the filename is the IP address encoded into hex. printf(1) or printf(3) can be nice for converting to/from HEX (hint: %X is unsigned hex (with capital letters) and %u is unsigned integer :)

And, for completeness sake, lets look at these:
/tftpboot/pxelinux.cfg/C000025
This corresponds to any of the 16 IP addresses:
192.0.2.80 192.0.2.81 192.0.2.82 192.0.2.83 192.0.2.84 192.0.2.85 192.0.2.86 192.0.2.87 192.0.2.88 192.0.2.89 192.0.2.90 192.0.2.91 192.0.2.92 192.0.2.93 192.0.2.94 192.0.2.95

/tftpboot/pxelinux.cfg/C00002
This is any of the IP addresses in 192.0.2.0/24.

/tftpboot/pxelinux.cfg/C0000
This is any of the IP addresses in 192.0.0.0/20.
/tftpboot/pxelinux.cfg/C000
This is any of the IP addresses in 192.0.0.0/16.
/tftpboot/pxelinux.cfg/C00
This is any of the IP addresses in 192.0.0.0/12.
/tftpboot/pxelinux.cfg/C0
This is any of the IP addresses in 192.0.0.0/8.
/tftpboot/pxelinux.cfg/C
This is any of the IP addresses in 192.0.0.0/4.
/tftpboot/pxelinux.cfg/default
This is, obviously, the default filename which any client will load.

So, I'm gen into *controlling* what is going on with each individual machine, so I don't see a whole lot of use for any of the possible filenames except for those based on the MAC or the IP or the "default". I especially find the "default" handy for "registering" new machines in some way and other things when you don't know enough about the machine to be sure of what to do!

One of these days I'll get around to writing something about PXELINUX configuration files themselves...

Profile

jimx0r: (Default)
jimx0r

July 2012

S M T W T F S
1234567
891011121314
15161718192021
22232425262728
2930 31    

Most Popular Tags

Style Credit

Expand Cut Tags

No cut tags
Page generated Jul. 24th, 2017 04:32 am
Powered by Dreamwidth Studios