projects::arch linux on ec2

summary

Arch Linux's minimalistic philosophy and high degree of customizability makes it a great choice for compute cloud deployment. It's light, fast, and scalable.

I am doing releases of Amazon EC2 AMIs based on Arch Linux roughly once per month.

contact

IRC Join us on irc.freenode.net in #archlinux-ec2!

If you find a bug or have any other comments, please send me an email, or ping me on twitter.

NOTE: This project is not run, sponsored, or endorsed by my employer or the Arch Linux project.

current releases

Release 2020.10.19

HVM Images

region ebs
hvm
x86_64
lts
s3
hvm
x86_64
lts
ebs
hvm
x86_64
stable
s3
hvm
x86_64
stable
us-east-1 ami-0dc324b29d22a7701 ami-008526933ddcf721b ami-024bd986c63802352 ami-0041bb885e3e0d026
us-east-2 ami-043b666ec218ceb75 ami-01e082ebc4e5e97c5 ami-075773610f44eff32 ami-0c5718d687ddb25bf
us-west-1 ami-089e477900c17154e ami-0c5799811725d84f2 ami-005fc732cc658b7dc ami-028fbc55064d91c0b
us-west-2 ami-001c701f04965b607 ami-0bc116e0986a4ae14 ami-04029944c9c31ba20 ami-00402231b9c3794c6
ca-central-1 ami-0e7048b2b6a998551 ami-0e9581e002ec17853 ami-0d3a512c7c6373774 ami-0174b56a2f989a4f9
eu-central-1 ami-080372577337ae6b5 ami-0a64abe7607e8bd74 ami-0e1cfe98bd83ed1a7 ami-019a90b59d2acf12e
eu-north-1 ami-00b1ae6761b832c3a ami-030772bee4a8f2162 ami-0ae68298e4168b334 ami-08bc74b45b33cc804
eu-west-1 ami-03a66eeeb695afb2c ami-04cabe582a15d197b ami-04e028d6839279a0d ami-01a422480e1368849
eu-west-2 ami-0294b600ed39bfc95 ami-08b9ba37c649ea1d9 ami-088ecb98e63472930 ami-0820ede41a99209fc
eu-west-3 ami-0a7493c630ee2d8bb ami-006334ce484971290 ami-0203c0de7f5118cfa ami-006482fa0e134c145
eu-south-1 ami-00595491df09a10c3 ami-00ed4ad4b6d8b93bc
af-south-1 ami-07eb3faff23725042 ami-09330cc6a115a103b
ap-east-1 ami-a2c183d3 ami-49c18338
ap-northeast-1 ami-0a6cd3d8a4d593d57 ami-09d9c5c7a01190574 ami-0e8131fc70cb5237c ami-0b09fac829b742bba
ap-northeast-2 ami-00035cbb759a3eb5b ami-015bed46649af534f ami-07880b09cd4bb3554 ami-0db86e9d2f1015b94
ap-south-1 ami-0a6e4f4c0ebf2950f ami-0210865678feb18a3 ami-0ca616d15c06bfe4c ami-0f2d45eaa6911d8fd
ap-southeast-1 ami-03971647f6ff561b1 ami-0aabac5392f18eceb ami-078d27a7359e30648 ami-0ab8270b562a1a3e5
ap-southeast-2 ami-0efa6ff4d106c93f5 ami-09aa9849db392d142 ami-0fbec04baf4214b6c ami-04d0e532a27b39dc8
sa-east-1 ami-05ad762feadd44c67 ami-099f6719865fa8766 ami-0730d0301e38aa1b0 ami-096839c40502ed3d3
me-south-1 ami-084a73945af8792f4 ami-09a7b7d4042c4fdf5

Release 2020.09.03

HVM Images

region ebs
hvm
x86_64
lts
s3
hvm
x86_64
lts
ebs
hvm
x86_64
stable
s3
hvm
x86_64
stable
us-east-1 ami-0687e03b02803dc18 ami-09fe68d23073fd348 ami-04f3e9f7a7690703d ami-0a9114ce779e84d37
us-east-2 ami-057b556e059980ae0 ami-0c2f1429c12a0041f ami-0f287d5fd2e5c1996 ami-0acb70a9e55cb9747
us-west-1 ami-0e66ffb9e61c36ebf ami-079cca86196d35a4f ami-0ecd59b86811f4a19 ami-0bc68832a5b8a0715
us-west-2 ami-087e5cb8e14e7e2de ami-047e440b5e087da7c ami-0ae280e1fdec63abc ami-05b45fb6b9247648c
ca-central-1 ami-0c0f9ea67a9226ec2 ami-024bc3be9005d2d7a ami-08606e99465ec76d4 ami-0bed51438d3a40a7b
eu-central-1 ami-07e7f5b7977ec18c6 ami-01632e810733efbec ami-0445a5a449f91d124 ami-01e0a75b991526163
eu-north-1 ami-072bb1136988e7d64 ami-05157080daf91d355 ami-0c5f133ed63802f6e ami-00d6dc3281ffd2ef8
eu-west-1 ami-0a2d714020d01a9a5 ami-08545803fe2822dc6 ami-0d04f7ce30d438df6 ami-0f2712849c73b9174
eu-west-2 ami-0cc05dfd64a0bf9db ami-0987c5c5e4362983a ami-01c907c055a88deee ami-07fa6fe6e06475b45
eu-west-3 ami-0c6ed2064e586d894 ami-048480ec0023e192e ami-0c2d66f9265ddd94a ami-0895d6ec156cdbb42
eu-south-1 ami-059debebe9e73adad ami-030af451bbd5e64ca
af-south-1 ami-002ec8e70890a43df ami-07671a77dce6d3061
ap-east-1 ami-40f5b631 ami-dcf9baad
ap-northeast-1 ami-0478b14e262d86ace ami-09a99b373e311a5de ami-0ace7116f67515e1c ami-0892c7912957e12d3
ap-northeast-2 ami-0bb00378d4e2abd4e ami-0d0155a6a5d127d21 ami-0f9f149db6a0fdef7 ami-0d2bf131e1a9d6c1e
ap-south-1 ami-0ea669c9fbca024f9 ami-0e8431f3d75edbcd1 ami-07589b44a5a55c3b3 ami-06e270b1f0573eb40
ap-southeast-1 ami-0a5d37b94bf58d293 ami-0034672f98aecd5f2 ami-0337e9b63b86cb428 ami-076417a6341976a7b
ap-southeast-2 ami-0ca0cd4f20e7eda18 ami-00bb3c6e5c0e34aa1 ami-0ad7522f7cf50b11d ami-00539fe2d1e50b57a
sa-east-1 ami-0a49d2fd91264e557 ami-0da793fcc2ac7fe34 ami-06cc60496c06fefe8 ami-0932bdd2fdb041ff8
me-south-1 ami-0e0d2b29d30bc8183 ami-085c76f049b90f56c

tools

The EC2 image build process is public, but the AMI registration portion is not. Here are the necessary tools to create an image file, but see the 2013-05-26 news post for information on how to register the images in EC2.

  • ami-build-backend - These files are held on the PXE server, and fetched when the guest boots.
  • ami-builder-image - This is a fork of archiso with some changes to automatically pull down my install script and do a few other things.
  • ec2-packages - These are the sources for all the packages contained in the 'ec2' Pacman repository.

recent changes and news

  • 2017-01-13

    I've added images for eu-west-2, ca-central-1, and us-east-2 as of today's release!

  • 2016-10-06

    The paravirtual images aren't booting properly in some regions, apparently because PV-GRUB is failing to load /boot/grub/menu.lst.

    Since PV is deprecated by AWS anyway, I'm going to stop making paravirtual AMIs from now on.

  • 2016-01-03

    Whoops. I didn't notice my GPG key was going to expire on January 2nd. My bad. I've updated the keypair and submitted it to pgp.mit.edu as well as created a new ec2-keyring package. Unfortunately to install the new keyring package you need to manually unbreak your Pacman keyring by fetching the updated key directly from the keyserver:

    # pacman-key -r A7B30DB9
    gpg: requesting key A7B30DB9 from hkp server pool.sks-keyservers.net
    gpg: key A7B30DB9: "Steven Noonan <steven@uplinklabs.net>" 5 new user IDs
    gpg: key A7B30DB9: "Steven Noonan <steven@uplinklabs.net>" 42 new signatures
    gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model
    gpg: depth: 0  valid:   1  signed:   7  trust: 0-, 0q, 0n, 0m, 0f, 1u
    gpg: depth: 1  valid:   7  signed:  66  trust: 1-, 0q, 0n, 6m, 0f, 0u
    gpg: depth: 2  valid:  66  signed:   6  trust: 66-, 0q, 0n, 0m, 0f, 0u
    gpg: next trustdb check due at 2016-01-22
    gpg: Total number processed: 1
    gpg:           new user IDs: 5
    gpg:         new signatures: 42
    ==> Updating trust database...
    gpg: next trustdb check due at 2016-01-22

    Once you do the above, you should be able to "pacman -Syu" as normal.

  • 2015-07-30

    The Arch Linux AMIs have been moved to a different AWS account. This unfortunately means that all the S3 buckets had to be recreated, and when that happened the old S3 endpoints stopped working correctly. Since pacman doesn't know how to handle HTTP 301 redirects, you're going to have to manually update any existing Arch Linux instances you have. To do this, need to change your /etc/pacman.conf repo path from this:

    [ec2]
    SigLevel = PackageRequired
    Server = https://s3.amazonaws.com/arch-linux-ami/repo/$arch

    to this:

    [ec2]
    SigLevel = PackageRequired
    Server = https://arch-linux-ami.s3.amazonaws.com/repo/$arch

    Be sure to force an update of the package databases once you're done:

    # pacman -Syy

    I've also stopped producing AMIs for the Beijing, China (cn-north-1) region for the moment. If someone has an account in that region and wishes to produce the AMIs for me there, please get ahold of me and we'll work on making it happen.

  • 2014-07-26

    I've added AMIs for the Beijing, China (cn-north-1) region.

  • 2014-06-27

    We're now up to Linux 3.15.2. I've removed xen-fbfront from the initramfs, because the module was causing 30-second boot delays:

    [    2.050081] tsc: Refined TSC clocksource calibration: 2793.267 MHz
    [    6.370066] xenbus_probe_frontend: Waiting for devices to initialise: 25s...20s...
    [   12.390306] random: nonblocking pool is initialized
    [   16.370070] 15s...10s...5s...0s...
    [   31.371241] xenbus_probe_frontend: Timeout connecting to device: device/vfb/0 (local state 3, remote state 1)
    

    The module is not required for an instance to boot correctly, so it can be removed from the initramfs. If you are running an AMI older than the 2014.06.27 release and would like to improve your instance's boot time, you can prune the module yourself:

    [root@ip-10-0-155-25 ~]# grep fbfront /etc/mkinitcpio.conf 
    MODULES="button ipmi-msghandler ipmi-poweroff virtio virtio-blk virtio-net virtio-pci virtio-ring
    xen-blkfront xen-fbfront xen-netfront xen-pcifront xen-privcmd hv_storvsc hv_balloon
    hv_vmbus hv_utils hv_netvsc ixgbevf"
    [root@ip-10-0-155-25 ~]# sed -ri 's/xen-fbfront //g' /etc/mkinitcpio.conf 
    [root@ip-10-0-155-25 ~]# mkinitcpio -p linux-ec2
    ==> Building image from preset: /etc/mkinitcpio.d/linux-ec2.preset: 'default'
      -> -k /boot/vmlinuz-linux-ec2 -c /etc/mkinitcpio.conf -g /boot/initramfs-linux-ec2.img -S autodetect
    ==> Starting build: 3.15.2-1-ec2
      -> Running build hook: [base]
      -> Running build hook: [udev]
      -> Running build hook: [modconf]
      -> Running build hook: [block]
      -> Running build hook: [filesystems]
      -> Running build hook: [growfs]
      -> Running build hook: [keyboard]
      -> Running build hook: [fsck]
    ==> Generating module dependencies
    ==> Creating gzip initcpio image: /boot/initramfs-linux-ec2.img
    ==> Image generation successful
    [root@ip-10-0-155-25 ~]#
    

    After removing xen-fbfront from mkinitcpio.conf's MODULES section, subsequent reboots will be 30 seconds shorter. Before:

    [root@ip-10-0-155-25 ~]# systemd-analyze
    Startup finished in 32.530s (kernel) + 3.997s (userspace) = 36.528s

    After:

    [root@ip-10-0-155-25 ~]# systemd-analyze
    Startup finished in 2.345s (kernel) + 2.092s (userspace) = 4.438s
  • 2014-06-19

    The AMI now uses systemd's networkd, timesyncd, and resolved services. This makes the AMI have a significantly smaller footprint. Right now our biggest non-core packages are CUDA (in the GPU AMI) and cloud-init, which has a large dependency chain. I'd like to slim things even further, but I'll need to investigate how to do so.

  • 2014-03-24

    We're up to Linux 3.13.7 for the ec2 kernel and 3.10.34 for the ec2-lts kernel. I didn't make a news post earlier, but kernels are now built with 'debug' and 'strip' options, which will create split-out debug information packages (i.e. linux-ec2-debug, linux-ec2-lts-debug). This is useful for tools like perf, oprofile, and systemtap. Note that the -debug packages are compressed with 'lrzip'. New AMI builds have lrzip preinstalled, but if you're running an instance based on one of the older AMIs, you will need to install lrzip before you can make use of the -debug packages.

  • 2013-11-28

    New AMIs are being built right now and contain a couple changes:

    • EBS root volumes are now automatically resized to fill the block device. You can take advantage of this feature by launching an instance with a root volume size larger than the snapshot.
    • The resolv.conf file permissions are now 0644, allowing non-root users to resolve hostnames.

  • 2013-11-26

    Geoff H. and David B. both reported an issue with the current AMI release. The /etc/resolv.conf permissions are set to 0600 rather than 0644, which means that non-root users cannot resolve hostnames to IP addresses. This is an unintentional regression, most likely caused by a default 'umask' change in some package. dhclient will create a new resolv.conf and copy it over any existing file, which preserves the target file's permissions. But if no such file exists, then the permissions of the source file are copied. Previously, this worked fine because the file was generated with 0644 permissions, but now it's being generated with 0600. I've implemented a fix for future AMI builds. In the meantime, if non-root users need to perform DNS requests in your instances, be sure to do 'chmod 0644 /etc/resolv.conf'.

  • 2013-11-06

    A new ec2-pacman-mirrors package is available, and will provide your instances with optimal Arch Linux mirrors for your EC2 region. The upgrade path is as follows:

    1. Edit /etc/pacman.conf, change 'ec2' mirror URL to https://s3.amazonaws.com/arch-linux-ami/repo/$arch
    2. Run 'pacman -Sy ec2-pacman-mirrors'

    New AMIs will be published very shortly which use the new mirror list and point to the new EC2 package repository.

  • 2013-05-26

    I've added some links to this page, which are the complete set of files needed to do an EC2 image build. This does not include the AMI registration process, however. The tools Amazon provides for HVM AMI registration are still under NDA at the moment, and the bits necessary to do that are included in my AMI registration tools. So I can't make those public right now. The process itself can be replicated relatively easily, though:

    1. Build your VM image using the build-backend and builder-image repos above. PXE is what I use, but you could just as easily make it into an ISO or something. If you intend to do an S3-backed AMI, you will need to make the image no larger than 10GB (I use 8GB).
    2. Trim the image down (I do a 'mount -o loop,discard' on the image, then 'fstrim' the mount point, making the image into a sparse file).
    3. Tarball the image (tar cSzf, S to preserve the sparseness).
    4. Upload the tarball to S3.
    5. In each region, launch an instance and attach an empty 8GB EBS volume to them.
    6. On each of those instances, download the tarball and extract with 'tar xSf'.
    7. Use 'ddpt', an enhanced dd which pays attention to the sparseness of the image, to copy the raw image file into the EBS device. I use "ddpt if=<imagefile> of=/dev/xvdf bs=512 conv=sparse oflag=sparse,fsync". The sparseness aspect is important, because otherwise you're copying empty blocks onto the EBS device, which makes the snapshot take much longer, and is really just a waste of time. EBS volumes already read-as-zero, so there's no sense copying zero blocks.
    8. Detach the EBS volume and terminate the instances.
    9. Snapshot the EBS volume.
    10. Delete the volume (not needed now).
    11. Use ec2-register to create an AMI from the snapshotted volume.

  • 2013-03-22

    I've started creating AMIs which have CUDA preinstalled. These are for the cg1.4xlarge instance type.

  • 2013-02-05

    Nothing too exciting lately. Today's release has Linux 3.7.6.

  • 2012-11-22

    New AMI releases, now with cloud-init. Thanks to Jeremy D for contributing his time and effort to making cloud-init work well on Arch Linux.

  • 2012-11-12

    Released new AMIs, primarily for the new AWS region in Sydney, Australia (ap-southeast-2).

  • 2012-11-08

    Today's AMIs are released. Nothing too fancy in this build: just updated packages, including linux-ec2 3.6.6-1.

  • 2012-10-21

    I've added a new linux-ec2 package which contains a patched v3.6.2 kernel. There are a few major differences between this kernel and the Arch Linux stock kernel:

    • Hangs on Xen fixed (patches from 3.6.3 stable-queue).
    • CONFIG_PREEMPT_VOLUNTARY instead of CONFIG_PREEMPT, this will allow for better scheduling as a domU.
    • CONFIG_HZ=100 instead of CONFIG_HZ=300, this allows for better performance on many-CPU instances, as there are fewer timer interrupts to preempt other tasks.
    • Many drivers removed, particularly those that didn't make sense for running in an EC2 instance. I've left drivers for my own hardware so I can experiment with it as a dom0 kernel as well. The kernel size is roughly half the stock Arch Linux kernel due to the stripped drivers.

    I am also building new AMIs right now, and am beating the i386 AMIs into working order. Once done I'll publish the next release (which should be 2012.10.21). Once it's available, it will show in the tables above.

  • 2012-10-16

    Do not upgrade HVM instance kernels to anything between 3.6.0 and 3.6.2 inclusive. You must wait for 3.6.3 or else your instance will not boot. We're currently waiting on this patch to be integrated into the mainline stable tree. This is also why I am probably not doing an AMI release this week, as the HVM AMIs would be totally broken.

    I've also taken a look at building i386 (well, i686) AMIs. I'm not really sure that it's worth the effort. Nobody really uses 32-bit AMIs anymore, and we'd need to fork the kernel just to make it happen. For now, i686 is on ice.

release notes

These AMIs are as close to a "vanilla" install as I can make them without making them functionally impaired on EC2. But here's the complete list of differences between the EC2 builds and a stock install:

  • High performance kernel specifically for EC2, including paravirtualization support on i386 and x86_64 AMIs, and more Xen-friendly process scheduling.

  • Kernel modules included in initrd, some of which are relevant outside of EC2 contexts (e.g. if you want to run the image in a non-EC2 environment such as KVM or Hyper-V):

    • KVM: virtio virtio-blk virtio-net virtio-pci virtio-ring
    • Xen: xen-blkfront xen-netfront xen-pcifront xen-privcmd
    • Hyper-V: hv_storvsc hv_balloon hv_vmbus hv_utils hv_netvsc
    • IPMI (e.g. EC2 reboot request): button ipmi-msghandler ipmi-poweroff
    • EC2 enhanced networking SR-IOV driver: ixgbevf
  • Extra packages installed: audit, cloud-init, ec2-keyring, ec2-pacman-mirrors, irqbalance, lrzip, openssh, rng-tools, rsync, systemd-sysvcompat

  • Added an extra package source for ec2-specific packages. The repository currently contains numerous packages useful on EC2. You can view the list of packages by doing 'pacman -Sy; pacman -Ss | grep ^ec2'

  • Additional services enabled at boot: rngd, sshd, cloud-init, irqbalance, auditd, systemd-networkd, systemd-timesyncd, systemd-resolved

  • User's public key is pulled from the EC2 instance metadata service at startup, and added to /root/.ssh/authorized_keys

  • SSH configured with 'PasswordAuthentication no', enforcing public key authentication

  • pacman loads (and automatically lsigns) the 'archlinux' and 'ec2' keyrings on the first boot (the latter keyring contains my public key used for package signing in the ec2 repo).

  • pacman mirror list is automatically selected at boot based on a list I created (based on rankmirrors run on instances in each region). These lists are provided by the package ec2-pacman-mirrors, which is in the ec2 repo.

  • dhclient is used instead of dhcpcd for robustness reasons. I found that dhcpcd gave up too quickly if it tried to do a DHCPREQUEST when the vif wasn't completely up, making the EC2 instance inaccessible.

  • dhclient is configured to retry forever, and request the following dhcp options: subnet-mask, broadcast-address, time-offset, routers, domain-name, domain-name-servers, host-name, interface-mtu, fqdn

  • /usr/bin/pinentry is symlinked to /usr/bin/pinentry-curses instead of the default pinentry-gtk, since gtk isn't available in this install and the primary access method is SSH.