Rebuilding the Freedesktop.org Server Complex

Ok, so it's really only two machines at present, but we have big plans.

As Daniel described in some detail, the machine was compromised and we've spent the last week rebuilding it. But, I wanted to describe our current setup and what we'd like to see in the near future.

The worst effect of the breakin was to disable all of our services, from web to email and even CVS. That's clearly not acceptable, and I think we can expect future security issues with our vast collection of random web services. This points to the usual solution -- use separate boxes for separate services. Based on conversations with other people running similar sites, I think we need:

  • A Web hosting machine. Running apache2 and as little else as possible. User accounts would normally have access to this machine.
  • A CVS machine. User accounts would not normally have access to this machine. CVS access is via SSH only. Project admins would have shell access to this machine to fix CVS repositories.
  • An Email machine. Sysadmin access only.
  • A security bastion for LDAP. LDAP updates can be managed by GPG-signed emails. Again, sysadmin access only.

We've got three machines available at present:

  • gabe, aka freedesktop.org. This is the HP server that Jim and I wrangled into our budget last year. Dual 2.8G Xeon processors with 512K caches (the 1M cache chips were not available). 3.5G of memory, six 38G scsi disks in a hardware raid array. The only "real" server we own, it's currently serving as web, mail and cvs server.

  • tycho, aka fooishbar.org. This is an older dual PIII server that I got from VA Linux Systems back in the day. It's got a couple of reasonable disks, but no RAID setup. It's certainly not powerful enough for either HTTP or CVS. But, it should make a fine mail server or perhaps LDAP. It needs a reinstall job, and we need to kick daniels off of it.

  • kara. This is a recent addition, a dual opteron box with IDE drives. It's really a desktop machine, and has the (current) distinct disadvantage of not being rebootable. We're using it as the main LDAP server for now, but I think that better be temporary until we can get tycho rebuilt.

We've stuck in a budget request for another "real" server. If that happens we can use that for CVS. It would be nice to get another small server for mail or ldap so that we could avoid using the dual opteron box; it's a desktop machine and isn't likely to be as reliable as we'd like to see.

X.org is also looking into building another site; I imagine we'll be able to piggy-back on that effort and mirror the fd.o content whereever x.org lands.