How can I add a resizable disk to my EC2 server?

Discussion

EC2 Volumes CANNOT be resized. If you require the flexibility of resizing your mounted Linux data disks on-the-fly, while the server is running, you can use Linux’s “built-in” LVM subsystem to make a concatenated disk composed of multiple virtual disks, each an EBS volume. When you need more space, you just allocate an additional EBS volume and add it to the RAID0 virtual disk real-time, no reboot required.

Procedure

Prerequisites

You will need the LVM commands (apt-get install lvm2).

Make a data disk

We start by making a new 100gig volume in Amazon’s Elastic Block Storage (EBS) system using the EC2 Command Line Tools:

$ ec2-create-volume --region us-east-1 -O AKIAXXXXXXXXXXXXXXXX -W nXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXA --availability-zone us-east-1a -s 100 --type standard

VOLUME  vol-beef08ba    100             us-east-1a      creating        2013-12-16T22:12:06+0000        standard
$

Note the new volume’s resource id (vol-beef08ba). We should tag the new volume with a Name:

$ ec2-create-tags --region us-east-1 -O AKIAXXXXXXXXXXXXXXXX -W nXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXA vol-beef08ba -t "Name=Data disk for New Server"
TAG     volume  vol-935b08ba    Name    Data disk for New Server
$

And we need to attach the volume to our new server as /dev/xvdf:

$ ec2-attach-volume --region us-east-1 -O AKIAXXXXXXXXXXXXXXXX -W nXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXA vol-beef08ba -i  i-beefef11 -d /dev/xvdf
ATTACHMENT      vol-beef08ba    i-beefef11      /dev/xvdf       attaching       2013-12-16T22:22:09+0000
$

It will attach to the new server as a plain, blank, virtual hard disk within seconds.

Log In

SSH Session

$ ssh -i .ssh/mykey.pem.txt ec2-user@54.193.12.194

       __|  __|_  )
       _|  (     /   Amazon Linux AMI
      ___|\___|___|

https://aws.amazon.com/amazon-linux-ami/2013.09-release-notes/
[ec2-user@ip-10-201-68-4 ~]$

Set up LVM

We need to configure Linux to use the extra disk (volume) that we created to be our data disk. We could just format it and mount it, but that would be too simple, right? So, instead we’ll set it up as a single-disk LVM volume management pool, so that we’ll be able to “grow” the data-disk easily later by just extending it with additional volumes.

Get a root shell:

[ec2-user@ip-10-201-68-4 ~]$ sudo bash
[root@ip-10-201-68-4 ec2-user]# id
uid=0(root) gid=0(root) groups=0(root)
[root@ip-10-201-68-4 ec2-user]#

Verify that the data disk (/dev/xvdf) is “physically” attached to the new server. A /dev/ node should have been automatically created for it:

[root@ip-10-201-68-4 ec2-user]# ls /dev/xvd*
/dev/xvda1  /dev/xvda3  /dev/xvdf
[root@ip-10-201-68-4 ec2-user]#

Initialize the volume for use by LVM:

[root@ip-10-201-68-4 ec2-user]# pvcreate /dev/xvdf
  Physical volume "/dev/xvdf" successfully created
[root@ip-10-201-68-4 ec2-user]#
Create a new volume group (we can call it "data") and add the disk to the VG:
SSH Session
[root@ip-10-201-68-4 ec2-user]# vgcreate data /dev/xvdf
  Volume group "data" successfully created
[root@ip-10-201-68-4 ec2-user]#

And cut out a logical volume (LV) from the VG that uses 100% of the available space in the VG:

[root@ip-10-201-68-4 ec2-user]# lvcreate -l 100%FREE -n data data
  Logical volume "data" created
[root@ip-10-201-68-4 ec2-user]#

(The “LV” is the blank RAID “disk” that we can format and use as we please.)

Format the LV (the RAID virtual drive) as a journaled (ext3) filesystem we can put our data in:

[root@ip-10-201-68-4 ec2-user]# mke2fs -j /dev/data/data
mke2fs 1.42.3 (14-May-2012)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
6553600 inodes, 26213376 blocks
1310668 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
800 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
        4096000, 7962624, 11239424, 20480000, 23887872

Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

[root@ip-10-201-68-4 ec2-user]#

It will take a minute or two to format.

Disable the max-mount-count and check-interval, so that the system doesn’t freeze-up for 10 minutes with a forced fsck when it’s rebooted:

[root@ip-10-201-68-4 ec2-user]# tune2fs /dev/data/data  -c -1 -i 0
tune2fs 1.42.3 (14-May-2012)
Setting maximal mount count to -1
Setting interval between checks to 0 seconds
[root@ip-10-201-68-4 ec2-user]#

Create a directory to be the mount-point and mount the formatted RAID drive!

[root@ip-10-201-68-4 ec2-user]# mkdir /data
[root@ip-10-201-68-4 ec2-user]# mount /dev/data/data /data
[root@ip-10-201-68-4 ec2-user]#

Verify the disk shows up as mounted:

[root@ip-10-201-68-4 ec2-user]# df -h /data
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/data-data
                       99G  188M   94G   1% /data
[root@ip-10-201-68-4 ec2-user]#

Set it up in /etc/fstab to automatically mount /data at boot-time:

[root@ip-10-201-68-4 ec2-user]# echo "/dev/data/data  /data  ext3  defaults 0 0" >> /etc/fstab
[root@ip-10-201-68-4 ec2-user]#

Troubleshooting

For LVM troubles, verify that the physical volume (PV) appears:

[root@ip-10-201-68-4 ec2-user]# pvdisplay
  --- Physical volume ---
  PV Name               /dev/sdf
  VG Name               data
  PV Size               100.00 GiB / not usable 4.00 MiB
  Allocatable           yes
  PE Size               4.00 MiB
  Total PE              25599
  Free PE               25599
  Allocated PE          0
  PV UUID               BAVBAK-h7mm-Nt1q-a2ot-4BLi-2Kda-mt2uFf

[root@ip-10-201-68-4 ec2-user]#

..and the volume group (VG):

[root@ip-10-201-68-4 ec2-user]# vgdisplay
  --- Volume group ---
  VG Name               data
  System ID
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  1
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                0
  Open LV               0
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               100.00 GiB
  PE Size               4.00 MiB
  Total PE              25599
  Alloc PE / Size       0 / 0
  Free  PE / Size       25599 / 100.00 GiB
  VG UUID               HBgGvb-aa1l-00OT-XRWT-FAgN-tGCX-iklHKk

[root@ip-10-201-68-4 ec2-user]#

..and the logical volume (the blank RAID “disk” that was formatted):

[root@ip-10-201-68-4 ec2-user]# lvdisplay
  --- Logical volume ---
  LV Path                /dev/data/data
  LV Name                data
  VG Name                data
  LV UUID                a9jBfe-L6xZ-2F4n-eF73-sBrW-1X1p-ayXCnq
  LV Write Access        read/write
  LV Creation host, time ip-10-201-68-4, 2013-12-16 22:33:21 +0000
  LV Status              available
  # open                 0
  LV Size                100.00 GiB
  Current LE             25599
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           253:0

[root@ip-10-201-68-4 ec2-user]#
Share: These icons link to social bookmarking sites where readers can share and discover new web pages.
  • Twitter
  • Facebook
  • Google Bookmarks
  • LinkedIn
  • Reddit
  • StumbleUpon

Leave a Reply

Your email address will not be published. Required fields are marked *