projects::arch linux on ec2

summary

Arch Linux's minimalistic philosophy and high degree of customizability makes it a great choice for compute cloud deployment. It's light, fast, and scalable.

I am doing releases of Amazon EC2 AMIs based on Arch Linux roughly once per month.

contact

IRC Join us on irc.freenode.net in #archlinux-ec2!

If you find a bug or have any other comments, please send me an email, or ping me on twitter.

NOTE: This project is not run, sponsored, or endorsed by my employer or the Arch Linux project.

current releases

Release 2021.01.14

HVM Images

region ebs
hvm
x86_64
lts
s3
hvm
x86_64
lts
ebs
hvm
x86_64
stable
s3
hvm
x86_64
stable
us-east-1 ami-0002ff9cc5c9b0d98 ami-0753b9015b63356ba ami-0ef0745adf93c94f7 ami-0025f4939ee5cbb7a
us-east-2 ami-0f0df120601069bf4 ami-09f20778db88ff8fd ami-02d88189f83c3f8cb ami-0ddc8740ac463e257
us-west-1 ami-0c756478e915b602e ami-0614d05b42f6f7943 ami-0a8201926ea73eee0 ami-098a74bb7d12c5518
us-west-2 ami-05edd6b1936260962 ami-0bf0f62effbd36ad3 ami-09ee139eb167e64a1 ami-0926be8f0f0597f5b
ca-central-1 ami-0160c5126c0781fe3 ami-0aea5770f529c7089 ami-0b9f417a7218eae74 ami-022654d8409d3b2e5
eu-central-1 ami-066024f62363b98ac ami-06675f64f853cc1f5 ami-007d97d2d7c425c68 ami-077aba539c08bec3f
eu-north-1 ami-0b2c1a297fc7a158f ami-0aeeb49e120933f82 ami-04e0aa2c342800291 ami-01301b3129eff57e3
eu-west-1 ami-091c4bd444fcac6a1 ami-01fb6d0860fbfb585 ami-0370aeae4564a0b10 ami-0fa1ae47d60c36d67
eu-west-2 ami-05fb9c1322f5e3f78 ami-058cde732b00a5e3e ami-0bb11eee7052e5560 ami-07932451568300598
eu-west-3 ami-0e9181171b3964d60 ami-082da3b63fcce72a7 ami-0d27a493ef7464778 ami-06d45327c6a2c7957
eu-south-1 ami-0a9d828db1c0c4acf ami-0c1c18d2da78f6f6b
af-south-1 ami-050b270741f072b5d ami-0836dbfbe81f057a7
ap-east-1 ami-068a730c25c83aafe ami-0207e5e3b643b95a4
ap-northeast-1 ami-0521069962a7d2592 ami-0261c770aa90c3fa2 ami-0cd1b519c2bd5b368 ami-07bd51e1b5cee37bf
ap-northeast-2 ami-0a6060d44e2a4ba82 ami-029e01ac8b11f3464 ami-01251d42814a7a4c2 ami-06ff78b41c75243cc
ap-south-1 ami-0463efcfc19b07c43 ami-03777899f6100aba8 ami-09b97cc31d2aa1395 ami-0a1d3eb39d9aea428
ap-southeast-1 ami-0c89a7445d559ec03 ami-053fa53cad3dc6e98 ami-0b7a85d653029b089 ami-007a1265572219f5b
ap-southeast-2 ami-04f5a3401b9b7c38c ami-031ceacce9768bf84 ami-0006c70e2255d6bd9 ami-026d0f3c5dc9088c9
sa-east-1 ami-00d571bb5f3c517fb ami-0e3f84eea9e7c5c2e ami-08841e33e853605d8 ami-09ec3fc4449fce906
me-south-1 ami-06b45898db861130a ami-0ad2b5b77203afced

Release 2020.12.28

HVM Images

region ebs
hvm
x86_64
lts
s3
hvm
x86_64
lts
ebs
hvm
x86_64
stable
s3
hvm
x86_64
stable
us-east-1 ami-061d1ac8982fd0a51 ami-097c622d4c609f5b0 ami-08ec5f2f33fe40a1b ami-0e5dc5b70b59ea1f4
us-east-2 ami-04e18562751b7599f ami-038a3ef2ef6b6fba0 ami-0c85e51bcb453585d ami-0e2a806c3caeb2a77
us-west-1 ami-06ec8697611b346e7 ami-0ef5221814623fc36 ami-03e49a73180832aa9 ami-0ba85cf7d0796cfe1
us-west-2 ami-0aca5fb7cdd56365d ami-0230b68a9725bc363 ami-0cfda0fc60e54b0d4 ami-01c573e07b1a48aff
ca-central-1 ami-04c86591916f8dd2c ami-0f2fa87f842b54a2e ami-0337adddb4b1d9167 ami-0e634dea465a331ed
eu-central-1 ami-04d408d142af7bf0f ami-06b1a74cad84ddbc9 ami-0e66895ec9d54f054 ami-0bc75f173253b87a4
eu-north-1 ami-0cb39a4ebc978f5d9 ami-0a3ee3c74a6f09cf4 ami-0525ed26220e840d8 ami-0d86af0e80dcce571
eu-west-1 ami-0fe30d04821db7ba7 ami-07d1ba2a381fb72ff ami-001d81491a5f758af ami-0b51e4b5611a85555
eu-west-2 ami-0b87a268324524c35 ami-055b732e88f20608b ami-0be2eb68918f2b549 ami-0530b79f75c44a30c
eu-west-3 ami-0f1363081ad37fc6f ami-0cabef6c086874cb0 ami-04a215beae684c38d ami-04cdff5b9f65d30d7
eu-south-1 ami-0868f256a77ee0db5 ami-0278dd1733f32029d
af-south-1 ami-0be8dd5ac73e74bdb ami-0f25689a0c5b5dd69
ap-east-1 ami-cf165abe ami-ce165abf
ap-northeast-1 ami-03babf37fb83a9298 ami-02c47af97e46d08c3 ami-036fb91ba1ebff087 ami-0472cb41077f4c8d7
ap-northeast-2 ami-025661c715e0f2f49 ami-0bc71670b8604fcd5 ami-0b799831bc69c211c ami-0ec8fe14e9ea040f0
ap-south-1 ami-0db20c5a45bdcd346 ami-029af2af0e8ef191e ami-011451aa923262baa ami-0b4edf7fbdfaad14e
ap-southeast-1 ami-02f41066c75ce9777 ami-0812fb38376192302 ami-0c482ecf474b10fa0 ami-01b9ce21c8a9a49c4
ap-southeast-2 ami-0dd9ad02c5a4da089 ami-0aa2d89a0fa462215 ami-07a30593638d59152 ami-02cc2ee677353fd49
sa-east-1 ami-068e5118015e79cce ami-00d2cc13e4ebc004f ami-094db800575cb96e4 ami-058bb8b3e870062ec
me-south-1 ami-0a76640f9b4ceef61 ami-033f06791a3dc7e91

tools

The EC2 image build process is public, but the AMI registration portion is not. Here are the necessary tools to create an image file, but see the 2013-05-26 news post for information on how to register the images in EC2.

  • ami-build-backend - These files are held on the PXE server, and fetched when the guest boots.
  • ami-builder-image - This is a fork of archiso with some changes to automatically pull down my install script and do a few other things.
  • ec2-packages - These are the sources for all the packages contained in the 'ec2' Pacman repository.

recent changes and news

  • 2017-01-13

    I've added images for eu-west-2, ca-central-1, and us-east-2 as of today's release!

  • 2016-10-06

    The paravirtual images aren't booting properly in some regions, apparently because PV-GRUB is failing to load /boot/grub/menu.lst.

    Since PV is deprecated by AWS anyway, I'm going to stop making paravirtual AMIs from now on.

  • 2016-01-03

    Whoops. I didn't notice my GPG key was going to expire on January 2nd. My bad. I've updated the keypair and submitted it to pgp.mit.edu as well as created a new ec2-keyring package. Unfortunately to install the new keyring package you need to manually unbreak your Pacman keyring by fetching the updated key directly from the keyserver:

    # pacman-key -r A7B30DB9
    gpg: requesting key A7B30DB9 from hkp server pool.sks-keyservers.net
    gpg: key A7B30DB9: "Steven Noonan <steven@uplinklabs.net>" 5 new user IDs
    gpg: key A7B30DB9: "Steven Noonan <steven@uplinklabs.net>" 42 new signatures
    gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model
    gpg: depth: 0  valid:   1  signed:   7  trust: 0-, 0q, 0n, 0m, 0f, 1u
    gpg: depth: 1  valid:   7  signed:  66  trust: 1-, 0q, 0n, 6m, 0f, 0u
    gpg: depth: 2  valid:  66  signed:   6  trust: 66-, 0q, 0n, 0m, 0f, 0u
    gpg: next trustdb check due at 2016-01-22
    gpg: Total number processed: 1
    gpg:           new user IDs: 5
    gpg:         new signatures: 42
    ==> Updating trust database...
    gpg: next trustdb check due at 2016-01-22

    Once you do the above, you should be able to "pacman -Syu" as normal.

  • 2015-07-30

    The Arch Linux AMIs have been moved to a different AWS account. This unfortunately means that all the S3 buckets had to be recreated, and when that happened the old S3 endpoints stopped working correctly. Since pacman doesn't know how to handle HTTP 301 redirects, you're going to have to manually update any existing Arch Linux instances you have. To do this, need to change your /etc/pacman.conf repo path from this:

    [ec2]
    SigLevel = PackageRequired
    Server = https://s3.amazonaws.com/arch-linux-ami/repo/$arch

    to this:

    [ec2]
    SigLevel = PackageRequired
    Server = https://arch-linux-ami.s3.amazonaws.com/repo/$arch

    Be sure to force an update of the package databases once you're done:

    # pacman -Syy

    I've also stopped producing AMIs for the Beijing, China (cn-north-1) region for the moment. If someone has an account in that region and wishes to produce the AMIs for me there, please get ahold of me and we'll work on making it happen.

  • 2014-07-26

    I've added AMIs for the Beijing, China (cn-north-1) region.

  • 2014-06-27

    We're now up to Linux 3.15.2. I've removed xen-fbfront from the initramfs, because the module was causing 30-second boot delays:

    [    2.050081] tsc: Refined TSC clocksource calibration: 2793.267 MHz
    [    6.370066] xenbus_probe_frontend: Waiting for devices to initialise: 25s...20s...
    [   12.390306] random: nonblocking pool is initialized
    [   16.370070] 15s...10s...5s...0s...
    [   31.371241] xenbus_probe_frontend: Timeout connecting to device: device/vfb/0 (local state 3, remote state 1)
    

    The module is not required for an instance to boot correctly, so it can be removed from the initramfs. If you are running an AMI older than the 2014.06.27 release and would like to improve your instance's boot time, you can prune the module yourself:

    [root@ip-10-0-155-25 ~]# grep fbfront /etc/mkinitcpio.conf 
    MODULES="button ipmi-msghandler ipmi-poweroff virtio virtio-blk virtio-net virtio-pci virtio-ring
    xen-blkfront xen-fbfront xen-netfront xen-pcifront xen-privcmd hv_storvsc hv_balloon
    hv_vmbus hv_utils hv_netvsc ixgbevf"
    [root@ip-10-0-155-25 ~]# sed -ri 's/xen-fbfront //g' /etc/mkinitcpio.conf 
    [root@ip-10-0-155-25 ~]# mkinitcpio -p linux-ec2
    ==> Building image from preset: /etc/mkinitcpio.d/linux-ec2.preset: 'default'
      -> -k /boot/vmlinuz-linux-ec2 -c /etc/mkinitcpio.conf -g /boot/initramfs-linux-ec2.img -S autodetect
    ==> Starting build: 3.15.2-1-ec2
      -> Running build hook: [base]
      -> Running build hook: [udev]
      -> Running build hook: [modconf]
      -> Running build hook: [block]
      -> Running build hook: [filesystems]
      -> Running build hook: [growfs]
      -> Running build hook: [keyboard]
      -> Running build hook: [fsck]
    ==> Generating module dependencies
    ==> Creating gzip initcpio image: /boot/initramfs-linux-ec2.img
    ==> Image generation successful
    [root@ip-10-0-155-25 ~]#
    

    After removing xen-fbfront from mkinitcpio.conf's MODULES section, subsequent reboots will be 30 seconds shorter. Before:

    [root@ip-10-0-155-25 ~]# systemd-analyze
    Startup finished in 32.530s (kernel) + 3.997s (userspace) = 36.528s

    After:

    [root@ip-10-0-155-25 ~]# systemd-analyze
    Startup finished in 2.345s (kernel) + 2.092s (userspace) = 4.438s
  • 2014-06-19

    The AMI now uses systemd's networkd, timesyncd, and resolved services. This makes the AMI have a significantly smaller footprint. Right now our biggest non-core packages are CUDA (in the GPU AMI) and cloud-init, which has a large dependency chain. I'd like to slim things even further, but I'll need to investigate how to do so.

  • 2014-03-24

    We're up to Linux 3.13.7 for the ec2 kernel and 3.10.34 for the ec2-lts kernel. I didn't make a news post earlier, but kernels are now built with 'debug' and 'strip' options, which will create split-out debug information packages (i.e. linux-ec2-debug, linux-ec2-lts-debug). This is useful for tools like perf, oprofile, and systemtap. Note that the -debug packages are compressed with 'lrzip'. New AMI builds have lrzip preinstalled, but if you're running an instance based on one of the older AMIs, you will need to install lrzip before you can make use of the -debug packages.

  • 2013-11-28

    New AMIs are being built right now and contain a couple changes:

    • EBS root volumes are now automatically resized to fill the block device. You can take advantage of this feature by launching an instance with a root volume size larger than the snapshot.
    • The resolv.conf file permissions are now 0644, allowing non-root users to resolve hostnames.

  • 2013-11-26

    Geoff H. and David B. both reported an issue with the current AMI release. The /etc/resolv.conf permissions are set to 0600 rather than 0644, which means that non-root users cannot resolve hostnames to IP addresses. This is an unintentional regression, most likely caused by a default 'umask' change in some package. dhclient will create a new resolv.conf and copy it over any existing file, which preserves the target file's permissions. But if no such file exists, then the permissions of the source file are copied. Previously, this worked fine because the file was generated with 0644 permissions, but now it's being generated with 0600. I've implemented a fix for future AMI builds. In the meantime, if non-root users need to perform DNS requests in your instances, be sure to do 'chmod 0644 /etc/resolv.conf'.

  • 2013-11-06

    A new ec2-pacman-mirrors package is available, and will provide your instances with optimal Arch Linux mirrors for your EC2 region. The upgrade path is as follows:

    1. Edit /etc/pacman.conf, change 'ec2' mirror URL to https://s3.amazonaws.com/arch-linux-ami/repo/$arch
    2. Run 'pacman -Sy ec2-pacman-mirrors'

    New AMIs will be published very shortly which use the new mirror list and point to the new EC2 package repository.

  • 2013-05-26

    I've added some links to this page, which are the complete set of files needed to do an EC2 image build. This does not include the AMI registration process, however. The tools Amazon provides for HVM AMI registration are still under NDA at the moment, and the bits necessary to do that are included in my AMI registration tools. So I can't make those public right now. The process itself can be replicated relatively easily, though:

    1. Build your VM image using the build-backend and builder-image repos above. PXE is what I use, but you could just as easily make it into an ISO or something. If you intend to do an S3-backed AMI, you will need to make the image no larger than 10GB (I use 8GB).
    2. Trim the image down (I do a 'mount -o loop,discard' on the image, then 'fstrim' the mount point, making the image into a sparse file).
    3. Tarball the image (tar cSzf, S to preserve the sparseness).
    4. Upload the tarball to S3.
    5. In each region, launch an instance and attach an empty 8GB EBS volume to them.
    6. On each of those instances, download the tarball and extract with 'tar xSf'.
    7. Use 'ddpt', an enhanced dd which pays attention to the sparseness of the image, to copy the raw image file into the EBS device. I use "ddpt if=<imagefile> of=/dev/xvdf bs=512 conv=sparse oflag=sparse,fsync". The sparseness aspect is important, because otherwise you're copying empty blocks onto the EBS device, which makes the snapshot take much longer, and is really just a waste of time. EBS volumes already read-as-zero, so there's no sense copying zero blocks.
    8. Detach the EBS volume and terminate the instances.
    9. Snapshot the EBS volume.
    10. Delete the volume (not needed now).
    11. Use ec2-register to create an AMI from the snapshotted volume.

  • 2013-03-22

    I've started creating AMIs which have CUDA preinstalled. These are for the cg1.4xlarge instance type.

  • 2013-02-05

    Nothing too exciting lately. Today's release has Linux 3.7.6.

  • 2012-11-22

    New AMI releases, now with cloud-init. Thanks to Jeremy D for contributing his time and effort to making cloud-init work well on Arch Linux.

  • 2012-11-12

    Released new AMIs, primarily for the new AWS region in Sydney, Australia (ap-southeast-2).

  • 2012-11-08

    Today's AMIs are released. Nothing too fancy in this build: just updated packages, including linux-ec2 3.6.6-1.

  • 2012-10-21

    I've added a new linux-ec2 package which contains a patched v3.6.2 kernel. There are a few major differences between this kernel and the Arch Linux stock kernel:

    • Hangs on Xen fixed (patches from 3.6.3 stable-queue).
    • CONFIG_PREEMPT_VOLUNTARY instead of CONFIG_PREEMPT, this will allow for better scheduling as a domU.
    • CONFIG_HZ=100 instead of CONFIG_HZ=300, this allows for better performance on many-CPU instances, as there are fewer timer interrupts to preempt other tasks.
    • Many drivers removed, particularly those that didn't make sense for running in an EC2 instance. I've left drivers for my own hardware so I can experiment with it as a dom0 kernel as well. The kernel size is roughly half the stock Arch Linux kernel due to the stripped drivers.

    I am also building new AMIs right now, and am beating the i386 AMIs into working order. Once done I'll publish the next release (which should be 2012.10.21). Once it's available, it will show in the tables above.

  • 2012-10-16

    Do not upgrade HVM instance kernels to anything between 3.6.0 and 3.6.2 inclusive. You must wait for 3.6.3 or else your instance will not boot. We're currently waiting on this patch to be integrated into the mainline stable tree. This is also why I am probably not doing an AMI release this week, as the HVM AMIs would be totally broken.

    I've also taken a look at building i386 (well, i686) AMIs. I'm not really sure that it's worth the effort. Nobody really uses 32-bit AMIs anymore, and we'd need to fork the kernel just to make it happen. For now, i686 is on ice.

release notes

These AMIs are as close to a "vanilla" install as I can make them without making them functionally impaired on EC2. But here's the complete list of differences between the EC2 builds and a stock install:

  • High performance kernel specifically for EC2, including paravirtualization support on i386 and x86_64 AMIs, and more Xen-friendly process scheduling.

  • Kernel modules included in initrd, some of which are relevant outside of EC2 contexts (e.g. if you want to run the image in a non-EC2 environment such as KVM or Hyper-V):

    • KVM: virtio virtio-blk virtio-net virtio-pci virtio-ring
    • Xen: xen-blkfront xen-netfront xen-pcifront xen-privcmd
    • Hyper-V: hv_storvsc hv_balloon hv_vmbus hv_utils hv_netvsc
    • IPMI (e.g. EC2 reboot request): button ipmi-msghandler ipmi-poweroff
    • EC2 enhanced networking SR-IOV driver: ixgbevf
  • Extra packages installed: audit, cloud-init, ec2-keyring, ec2-pacman-mirrors, irqbalance, lrzip, openssh, rng-tools, rsync, systemd-sysvcompat

  • Added an extra package source for ec2-specific packages. The repository currently contains numerous packages useful on EC2. You can view the list of packages by doing 'pacman -Sy; pacman -Ss | grep ^ec2'

  • Additional services enabled at boot: rngd, sshd, cloud-init, irqbalance, auditd, systemd-networkd, systemd-timesyncd, systemd-resolved

  • User's public key is pulled from the EC2 instance metadata service at startup, and added to /root/.ssh/authorized_keys

  • SSH configured with 'PasswordAuthentication no', enforcing public key authentication

  • pacman loads (and automatically lsigns) the 'archlinux' and 'ec2' keyrings on the first boot (the latter keyring contains my public key used for package signing in the ec2 repo).

  • pacman mirror list is automatically selected at boot based on a list I created (based on rankmirrors run on instances in each region). These lists are provided by the package ec2-pacman-mirrors, which is in the ec2 repo.

  • dhclient is used instead of dhcpcd for robustness reasons. I found that dhcpcd gave up too quickly if it tried to do a DHCPREQUEST when the vif wasn't completely up, making the EC2 instance inaccessible.

  • dhclient is configured to retry forever, and request the following dhcp options: subnet-mask, broadcast-address, time-offset, routers, domain-name, domain-name-servers, host-name, interface-mtu, fqdn

  • /usr/bin/pinentry is symlinked to /usr/bin/pinentry-curses instead of the default pinentry-gtk, since gtk isn't available in this install and the primary access method is SSH.