Centos AMI on Amazon EC2 – How To Resize The Disk

When using the official CentOS Linux AMI on EC2, it comes (currently) with only an 8GB partition.  Want to resize that?  Here is how…

First make sure you start a small (t2.micro or similar) instance to do this work with.  It should be based on the Amazon Linux AMI.  You should have root access to it.  Let’s call it “fix-the-disk-instance”.

  1. Stop the NEW instance
    1. Go to EC2 > Instances
    2. Select the Instance
    3. Make sure the Instance has a name that makes sense (e.g. snap.appcove.net)
    4. Click “Actions” > “Instance State” > Stop
  2. Detach the block device
    1. Go to EC2 > Volumes
    2. Find the volume by looking under Attachment Information for the instance name (e.g. snap.appcove.net)
    3. Select the volume
    4. Click “Actions” > “Detach Volume”
  3. Attach the block device to “fix-the-disk-instance”
    1. Go to EC2 > Volumes
    2. Find the volume by looking under Attachment Information for the instance name (e.g. snap.appcove.net)
    3. Select the volume
    4. Click “Actions” > “Attach Volume”
    5. Attach it to “fix-the-disk-instance” so we can fix it up
  4. Start “fix-the-disk-instance” and login
    1. Go to EC2 > Volumes
    2. Select the Instance “fix-the-disk-instance”
    3. Click “Actions” > “Instance State” > Start
    4. Wait for it to start
    5. SSH into it and escalate to root
  5. Run `parted` command and print list
    1. As root, run the following command: parted
    2. Type the following command: print all<enter>
    3. Find your disk in the output and note the path (e.g. /dev/xvdf)
    4. Quit the program by typing: quit<enter>
  6. Resize the partition:
    1. Run fdisk on the device path found above (e.g. fdisk /dev/xvdf)
    2. List the partitions
      1. Type: p<enter>
      2. Take note of the “Start” column (e.g. 2048)
    3. Delete the partition
      1. Type: d<enter>
    4. Create a new one by typing:
      1. n (for new)
      2. p (for primary)
      3. 1 (for first partition)
      4. Type first sector the same as the Start column above (e.g. 2048)
      5. Last sector should be the default (end of disk)
      6. w (for writing to disk)
  7. Now check the partition and resize it
    1. Run this command making sure to use the right device path (but add a 1 to the end for the first partition)
      1. e2fsck -f /dev/xvdf1
    2. Resize the partition making sure to use the right device path (but add a 1 to the end for the first partition)
      1. resize2fs /dev/xvdf1
  8. Shutdown the instance “fix-the-disk-instance”
    1. Go to EC2 > Instances
    2. Select the Instance “fix-the-disk-instance”
    3. Click “Actions” > “Instance State” > Stop
  9. Detach the block device and reattach it to the original server
    1. Go to EC2 > Volumes
    2. Find the volume by looking under Attachment Information for the instance name (e.g. fix-the-device-instance).
    3. Make sure it is the right volume by looking at the ID/Size.
    4. Select the volume
    5. Click “Actions” > “Detach Volume”
    6. Click “Actions” > “Attach Volume”
    7. IMPORTANT: enter “/dev/sda1” as the device so it attaches as the root volume
    8. Attach it to the original server
  10. Start “fix-the-disk-instance” and login
    1. Go to EC2 > Volumes
    2. Select the original instance (e.g. “snap.appcove.net”)
    3. Click “Actions” > “Instance State” > Start
    4. Wait for it to start

 

And that’s all!

 

 

2016-03-20 - B162957

2016-03-20 - B164242

2016-03-20 - B164338

 

 

I highly recommend yum + createrepo + rpmbuild

As I was discussing lightly before, I have recently been involved in building quite a few RPMs for our server clusters at AppCove.


Where we have arrived:

Our (new) primary production cluster consists of multiple RedHat Enterprise Linux 5 boxes in different capacities (webserver, appserver, database master, database slave, etc…).

Each machine is registered with 3 yum repositories:

  1. RHEL (RedHat Enterprise Linux)
  2. EPEL (Extra Packages for Enterprise Linux)
  3. ACN (AppCove Network)

All of our custom software packages and custom builds of open source software are placed into individual RPMs, and entered into our ACN repository.

From there, it is a snap to update any given server with the correct version of the software that server needs.

We have a dedicated build area, versioned with git, that is used to build and package all of the custom software that is needed.

(note, RPMs are not used for web application deployment — rsync via ssh is used for that)


Recommendation:

Having worked through the process from start to finish, I must say that I would highly recommend the following tools to anyone who is responsible for RedHat Enterprise, Centos, or Fedora system administration.

  • git – to keep your .spec files versioned
  • rpmbuild – to build the rpms
  • createrepo – to create your very own yum repository
  • apache – to serve the yum repository
  • yum – to obtain, install, and upgrade your rpms

Additionally, if you are using RedHat Enterprise or Centos, I would highly recommend using Extra Packages for Enterprise Linux (EPEL) to get a few of those “other” packages that don’t come with your OS (git, for example).


Learning how to build RPMs was a fairly steep curve.  But it wasn’t long.  It is one of those things that if you know it you say “that’s easy” and if you don’t you say “what the ???

yum+rpm was invented (I assume) to make life easier for countless system administrators and software publishers.  So it’s not the kind of thing that everyone is involved in.

I was a bit tough to figure out the caveats of how to correctly build RPM’s that work.  The documentation is a bit sparse.  A bit here and a bit there.


What are the benefits?

Many.  Let me list a few.

Your system stays really clean. With RPMs, you can uninstall everything you installed without leaving extra files laying around.

Upgrades are a snap. Once you have registered your own yum repository on a system, you can upgrade a given package by running:

yum upgrade your-package

All your systems can be on the same “page”. It is very easy, using yum, to ensure that all of your systems are using the exact same version of software.

Custom builds are super easy to maintain. We custom-compile php, python, and various other software.  Once the .spec files are in place, all of your software can be re-packaged with a single command.

In our specific case, we wanted to have the memcached client statically compiled into PHP.  With a few extra commands in the .spec file, it was a snap to pull in the source from pecl, and update `configure` to take it into account.

All builds can take place in one place. With one set of documentation, one consistent set of development tools, etc…  We have a user called `build` on one of the hosts that is specifically used for building all of the RPMs.


Where to learn?

The best way to learn, as usual, is to jump in and figure it out.   There is some really good documentation buried in the rpm.org site.   It is a book called Maximum RPM, origninally published by redhat.  The current snapshot of the book is available online.

http://www.rpm.org/max-rpm-snapshot/

Google is another good resource, depending on what it is you are looking for.

Update [1] on Fedora vs Redhat Enterprise Linux

This is in reference to https://blog.gahooa.com/2009/01/18/fedora-or-redhat-enterprise-linux-in-a-production-environment/.

After the excellent comment by Sergio Olivo, I did some heavy looking into the Extra Packages for Enterprise Linux project (EPEL for short).  On a brand-spanking-new RHEL 5 box, I installed the YUM repository for EPEL, and quite immediately had access to tons of extra packages.  Erlang is there.  Git is there.  Memcached is there.  Sweet!

However, EPEL does not update or replace the version of any packages provided by RHEL.

So the problem of having out of date versions of PHP and Python still remain.  Next I looked into using a third party RPM repository (provided by RackSpace).  They provide updated versions of PHP and a number of PHP modules.  But alas, this created incompatibilities with the EPEL packages for PHP.  This is because EPEL packages are targeted for RHEL versions. Bla…

So here is what I decided to do (haven’t done it yet, but will soon).  We will build and package our own custom set of RPMs for RHEL 5, and publish them in an RPM repository.  Then we will simply point each server to that repository in addition to the main RHEL repository, and poof, problem solved.  We may also use EPEL for things like Erlang and git.  Or we may compile from source.  Not sure.

For those of you who are not familiar with YUM RPM repositories, they can be as simple as a specail directory structure served by a webserver.

There are a few items remaining to be concluded, but they should fall into place fairly quickly.