Where we have arrived:
Our (new) primary production cluster consists of multiple RedHat Enterprise Linux 5 boxes in different capacities (webserver, appserver, database master, database slave, etc…).
Each machine is registered with 3 yum repositories:
- RHEL (RedHat Enterprise Linux)
- EPEL (Extra Packages for Enterprise Linux)
- ACN (AppCove Network)
All of our custom software packages and custom builds of open source software are placed into individual RPMs, and entered into our ACN repository.
From there, it is a snap to update any given server with the correct version of the software that server needs.
We have a dedicated build area, versioned with git, that is used to build and package all of the custom software that is needed.
Having worked through the process from start to finish, I must say that I would highly recommend the following tools to anyone who is responsible for RedHat Enterprise, Centos, or Fedora system administration.
- git – to keep your .spec files versioned
- rpmbuild – to build the rpms
- createrepo – to create your very own yum repository
- apache – to serve the yum repository
- yum – to obtain, install, and upgrade your rpms
Additionally, if you are using RedHat Enterprise or Centos, I would highly recommend using Extra Packages for Enterprise Linux (EPEL) to get a few of those “other” packages that don’t come with your OS (git, for example).
Learning how to build RPMs was a fairly steep curve. But it wasn’t long. It is one of those things that if you know it you say “that’s easy” and if you don’t you say “what the ???”
yum+rpm was invented (I assume) to make life easier for countless system administrators and software publishers. So it’s not the kind of thing that everyone is involved in.
I was a bit tough to figure out the caveats of how to correctly build RPM’s that work. The documentation is a bit sparse. A bit here and a bit there.
What are the benefits?
Many. Let me list a few.
Your system stays really clean. With RPMs, you can uninstall everything you installed without leaving extra files laying around.
Upgrades are a snap. Once you have registered your own yum repository on a system, you can upgrade a given package by running:
yum upgrade your-package
All your systems can be on the same “page”. It is very easy, using yum, to ensure that all of your systems are using the exact same version of software.
Custom builds are super easy to maintain. We custom-compile php, python, and various other software. Once the .spec files are in place, all of your software can be re-packaged with a single command.
In our specific case, we wanted to have the memcached client statically compiled into PHP. With a few extra commands in the .spec file, it was a snap to pull in the source from pecl, and update `configure` to take it into account.
All builds can take place in one place. With one set of documentation, one consistent set of development tools, etc… We have a user called `build` on one of the hosts that is specifically used for building all of the RPMs.
Where to learn?
The best way to learn, as usual, is to jump in and figure it out. There is some really good documentation buried in the rpm.org site. It is a book called Maximum RPM, origninally published by redhat. The current snapshot of the book is available online.
Google is another good resource, depending on what it is you are looking for.