projects::arch linux on ec2

summary

Arch Linux's minimalistic philosophy and high degree of customizability makes it a great choice for compute cloud deployment. It's light, fast, and scalable.

I am doing releases of Amazon EC2 AMIs based on Arch Linux roughly once per month.

contact

IRC Join us on irc.freenode.net in #archlinux-ec2!

If you find a bug or have any other comments, please send me an email, or ping me on twitter.

NOTE: This project is not run, sponsored, or endorsed by my employer or the Arch Linux project.

current releases

Release 2020.06.14

HVM Images

region ebs
hvm
x86_64
lts
s3
hvm
x86_64
lts
ebs
hvm
x86_64
stable
s3
hvm
x86_64
stable
us-east-1 ami-0d086d189e4c07fa2 ami-07b0f5bf626eddc13 ami-0904189dbef2a3722 ami-016b471f111547ab3
us-east-2 ami-067c7c620bc00297a ami-086f0f5a346ca39f1 ami-0732f750b255470d2 ami-0522689a0c2620dea
us-west-1 ami-028943629a1cf6cda ami-0c1c34fed60d54b6a ami-0395c3cd8d1e895da ami-0a675a72b5ba8bf4e
us-west-2 ami-0f649957d53fa914a ami-0019bf264a09b3c59 ami-02ac32f98bdd6605f ami-0154720f90ac7eefd
ca-central-1 ami-0a7961665967f5e55 ami-0934bbddc19c6e9e2 ami-0d7e8c2290f81f1e3 ami-03a8dc4a419abeece
eu-central-1 ami-0686bf0dd4c49a92f ami-0dfe997cb6275102b ami-086791b31ca9686a7 ami-06da92553674d466c
eu-north-1 ami-0b65f6e800b7971a9 ami-0148fcbd14dc0954f ami-08869f7a1600035b7 ami-0fcb5337e470783cd
eu-west-1 ami-090c6a408b07b2708 ami-04efa9d593f1f3d08 ami-0c5c7b7e492fab0b6 ami-0cbe96d23f97014e2
eu-west-2 ami-0a8649d77a0e08c7c ami-0ae1719953bd6c0c1 ami-0348964f8b834d145 ami-04ebaa62cd5dd5b48
eu-west-3 ami-0faed17141b5b83b1 ami-03924faadd01fb566 ami-0d4646ca215ed7047 ami-0d752d555e9e1afc9
eu-south-1 ami-0a98ec2ec212c774b ami-0edfb765ea5bc8762
af-south-1 ami-0d5b887f8c1eee5bc ami-088834a811dacc530
ap-east-1 ami-1e27676f ami-9d2767ec
ap-northeast-1 ami-0112d64779e258a0e ami-0524619a0959e916c ami-0960fb3c3a2edd6a0 ami-06563e6e03fcadaf1
ap-northeast-2 ami-002e74d00ef4419e0 ami-0186be1371b9dc67e ami-00572cd375f3e9b2e ami-04a5167654a778089
ap-south-1 ami-0d7f49350c17e4814 ami-099a27eefa736080f ami-0f6092f4a4b035fdb ami-0361d42965e27fecd
ap-southeast-1 ami-03702cf60e7f50704 ami-04961dbfdc4777db0 ami-08612b5ec7e5d4239 ami-0c9a12f79bbe8c852
ap-southeast-2 ami-0483c921c9dec59c3 ami-0fe2b260ddba24ba6 ami-04faf55bad2ec3d6b ami-0d829785dc905c5ba
sa-east-1 ami-07206c39670bf0d18 ami-0ecd0631f34882e88 ami-0951cfc7d036f77ab ami-009d85716b7a25891
me-south-1 ami-0e6c89cadf3ca2beb ami-044130053545e44a0

Release 2020.04.23

HVM Images

region ebs
hvm
x86_64
lts
s3
hvm
x86_64
lts
ebs
hvm
x86_64
stable
s3
hvm
x86_64
stable
us-east-1 ami-01d988fc8ed4684e6 ami-01f0af992c874c022 ami-05ceb2a010d3a3057 ami-0e85114988fc55458
us-east-2 ami-0af64ae30af94a194 ami-0e70eb51bcd8f9c0b ami-0dfef6137ada9bde4 ami-09e6b4b0506312758
us-west-1 ami-0da8fb36e600a2383 ami-038326b087494425d ami-0ae6570ce62c236c5 ami-05cbd6cbdb3d56b4b
us-west-2 ami-06fa871e02da09d8f ami-0de4eba8ffa13caf8 ami-0e65385d9994e0de2 ami-08ed93274b41b76ae
ca-central-1 ami-0ffa2056f11adb2ea ami-0bfb497c11274de42 ami-0b2ce84b3dd55099b ami-078131a300ebd2edc
eu-central-1 ami-03e8cd60c6e05ff60 ami-03cace6679721107d ami-0699b268e954e128a ami-0baa4aa8c176d32c2
eu-north-1 ami-0473a3b83fd2c8a12 ami-00e4f9184c8dd6942 ami-090925aed63eed783 ami-0ab58c4d7ded55b91
eu-west-1 ami-0c00b7634203d3a06 ami-030cce85f643b9212 ami-08b4188938de94a60 ami-01228b41d05be641f
eu-west-2 ami-0433699dff6375331 ami-006d1557d373c4820 ami-02e52f10e05bc1823 ami-0b1434832bd5ceed7
eu-west-3 ami-094eb7a76d291f77b ami-0701b1d1e4509bf49 ami-017a04ade8abfeaf9 ami-0998eb09ca5538b05
af-south-1 ami-08824370575f3c676 ami-0b3ecf9c0b0ca947d
ap-east-1 ami-eed3959f ami-f6d29487
ap-northeast-1 ami-0cd979d94de332814 ami-0608fa3dc999b0247 ami-0fcf6df96bd1453cc ami-0a7a2381ae48f31ff
ap-northeast-2 ami-0a1f8597c22d7352d ami-07e8d1e2e2920f7aa ami-0485a44bae44b56c8 ami-02d03511619763d05
ap-south-1 ami-035c12c37a16a7065 ami-060f4ad260bc7fcd4 ami-067e70d33f6bc3930 ami-0dc2ccda54324596b
ap-southeast-1 ami-0eab7ad743f42cce7 ami-07e34a1567def4018 ami-077593a96265b2dad ami-0e0ddc6cffd06a56c
ap-southeast-2 ami-0731be3f1f842a780 ami-0932f7e4e205561de ami-061dcd41b4fbae206 ami-0b5d020172457f429
sa-east-1 ami-07f99be1e9a08abe3 ami-0d016f536233096bd ami-046f49f9d9797bf2f ami-0f3a4b51b2c491d91
me-south-1 ami-0b6561a45d22f36be ami-0e4302d0515dc24cd

tools

The EC2 image build process is public, but the AMI registration portion is not. Here are the necessary tools to create an image file, but see the 2013-05-26 news post for information on how to register the images in EC2.

  • ami-build-backend - These files are held on the PXE server, and fetched when the guest boots.
  • ami-builder-image - This is a fork of archiso with some changes to automatically pull down my install script and do a few other things.
  • ec2-packages - These are the sources for all the packages contained in the 'ec2' Pacman repository.

recent changes and news

  • 2017-01-13

    I've added images for eu-west-2, ca-central-1, and us-east-2 as of today's release!

  • 2016-10-06

    The paravirtual images aren't booting properly in some regions, apparently because PV-GRUB is failing to load /boot/grub/menu.lst.

    Since PV is deprecated by AWS anyway, I'm going to stop making paravirtual AMIs from now on.

  • 2016-01-03

    Whoops. I didn't notice my GPG key was going to expire on January 2nd. My bad. I've updated the keypair and submitted it to pgp.mit.edu as well as created a new ec2-keyring package. Unfortunately to install the new keyring package you need to manually unbreak your Pacman keyring by fetching the updated key directly from the keyserver:

    # pacman-key -r A7B30DB9
    gpg: requesting key A7B30DB9 from hkp server pool.sks-keyservers.net
    gpg: key A7B30DB9: "Steven Noonan <steven@uplinklabs.net>" 5 new user IDs
    gpg: key A7B30DB9: "Steven Noonan <steven@uplinklabs.net>" 42 new signatures
    gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model
    gpg: depth: 0  valid:   1  signed:   7  trust: 0-, 0q, 0n, 0m, 0f, 1u
    gpg: depth: 1  valid:   7  signed:  66  trust: 1-, 0q, 0n, 6m, 0f, 0u
    gpg: depth: 2  valid:  66  signed:   6  trust: 66-, 0q, 0n, 0m, 0f, 0u
    gpg: next trustdb check due at 2016-01-22
    gpg: Total number processed: 1
    gpg:           new user IDs: 5
    gpg:         new signatures: 42
    ==> Updating trust database...
    gpg: next trustdb check due at 2016-01-22

    Once you do the above, you should be able to "pacman -Syu" as normal.

  • 2015-07-30

    The Arch Linux AMIs have been moved to a different AWS account. This unfortunately means that all the S3 buckets had to be recreated, and when that happened the old S3 endpoints stopped working correctly. Since pacman doesn't know how to handle HTTP 301 redirects, you're going to have to manually update any existing Arch Linux instances you have. To do this, need to change your /etc/pacman.conf repo path from this:

    [ec2]
    SigLevel = PackageRequired
    Server = https://s3.amazonaws.com/arch-linux-ami/repo/$arch

    to this:

    [ec2]
    SigLevel = PackageRequired
    Server = https://arch-linux-ami.s3.amazonaws.com/repo/$arch

    Be sure to force an update of the package databases once you're done:

    # pacman -Syy

    I've also stopped producing AMIs for the Beijing, China (cn-north-1) region for the moment. If someone has an account in that region and wishes to produce the AMIs for me there, please get ahold of me and we'll work on making it happen.

  • 2014-07-26

    I've added AMIs for the Beijing, China (cn-north-1) region.

  • 2014-06-27

    We're now up to Linux 3.15.2. I've removed xen-fbfront from the initramfs, because the module was causing 30-second boot delays:

    [    2.050081] tsc: Refined TSC clocksource calibration: 2793.267 MHz
    [    6.370066] xenbus_probe_frontend: Waiting for devices to initialise: 25s...20s...
    [   12.390306] random: nonblocking pool is initialized
    [   16.370070] 15s...10s...5s...0s...
    [   31.371241] xenbus_probe_frontend: Timeout connecting to device: device/vfb/0 (local state 3, remote state 1)
    

    The module is not required for an instance to boot correctly, so it can be removed from the initramfs. If you are running an AMI older than the 2014.06.27 release and would like to improve your instance's boot time, you can prune the module yourself:

    [root@ip-10-0-155-25 ~]# grep fbfront /etc/mkinitcpio.conf 
    MODULES="button ipmi-msghandler ipmi-poweroff virtio virtio-blk virtio-net virtio-pci virtio-ring
    xen-blkfront xen-fbfront xen-netfront xen-pcifront xen-privcmd hv_storvsc hv_balloon
    hv_vmbus hv_utils hv_netvsc ixgbevf"
    [root@ip-10-0-155-25 ~]# sed -ri 's/xen-fbfront //g' /etc/mkinitcpio.conf 
    [root@ip-10-0-155-25 ~]# mkinitcpio -p linux-ec2
    ==> Building image from preset: /etc/mkinitcpio.d/linux-ec2.preset: 'default'
      -> -k /boot/vmlinuz-linux-ec2 -c /etc/mkinitcpio.conf -g /boot/initramfs-linux-ec2.img -S autodetect
    ==> Starting build: 3.15.2-1-ec2
      -> Running build hook: [base]
      -> Running build hook: [udev]
      -> Running build hook: [modconf]
      -> Running build hook: [block]
      -> Running build hook: [filesystems]
      -> Running build hook: [growfs]
      -> Running build hook: [keyboard]
      -> Running build hook: [fsck]
    ==> Generating module dependencies
    ==> Creating gzip initcpio image: /boot/initramfs-linux-ec2.img
    ==> Image generation successful
    [root@ip-10-0-155-25 ~]#
    

    After removing xen-fbfront from mkinitcpio.conf's MODULES section, subsequent reboots will be 30 seconds shorter. Before:

    [root@ip-10-0-155-25 ~]# systemd-analyze
    Startup finished in 32.530s (kernel) + 3.997s (userspace) = 36.528s

    After:

    [root@ip-10-0-155-25 ~]# systemd-analyze
    Startup finished in 2.345s (kernel) + 2.092s (userspace) = 4.438s
  • 2014-06-19

    The AMI now uses systemd's networkd, timesyncd, and resolved services. This makes the AMI have a significantly smaller footprint. Right now our biggest non-core packages are CUDA (in the GPU AMI) and cloud-init, which has a large dependency chain. I'd like to slim things even further, but I'll need to investigate how to do so.

  • 2014-03-24

    We're up to Linux 3.13.7 for the ec2 kernel and 3.10.34 for the ec2-lts kernel. I didn't make a news post earlier, but kernels are now built with 'debug' and 'strip' options, which will create split-out debug information packages (i.e. linux-ec2-debug, linux-ec2-lts-debug). This is useful for tools like perf, oprofile, and systemtap. Note that the -debug packages are compressed with 'lrzip'. New AMI builds have lrzip preinstalled, but if you're running an instance based on one of the older AMIs, you will need to install lrzip before you can make use of the -debug packages.

  • 2013-11-28

    New AMIs are being built right now and contain a couple changes:

    • EBS root volumes are now automatically resized to fill the block device. You can take advantage of this feature by launching an instance with a root volume size larger than the snapshot.
    • The resolv.conf file permissions are now 0644, allowing non-root users to resolve hostnames.

  • 2013-11-26

    Geoff H. and David B. both reported an issue with the current AMI release. The /etc/resolv.conf permissions are set to 0600 rather than 0644, which means that non-root users cannot resolve hostnames to IP addresses. This is an unintentional regression, most likely caused by a default 'umask' change in some package. dhclient will create a new resolv.conf and copy it over any existing file, which preserves the target file's permissions. But if no such file exists, then the permissions of the source file are copied. Previously, this worked fine because the file was generated with 0644 permissions, but now it's being generated with 0600. I've implemented a fix for future AMI builds. In the meantime, if non-root users need to perform DNS requests in your instances, be sure to do 'chmod 0644 /etc/resolv.conf'.

  • 2013-11-06

    A new ec2-pacman-mirrors package is available, and will provide your instances with optimal Arch Linux mirrors for your EC2 region. The upgrade path is as follows:

    1. Edit /etc/pacman.conf, change 'ec2' mirror URL to https://s3.amazonaws.com/arch-linux-ami/repo/$arch
    2. Run 'pacman -Sy ec2-pacman-mirrors'

    New AMIs will be published very shortly which use the new mirror list and point to the new EC2 package repository.

  • 2013-05-26

    I've added some links to this page, which are the complete set of files needed to do an EC2 image build. This does not include the AMI registration process, however. The tools Amazon provides for HVM AMI registration are still under NDA at the moment, and the bits necessary to do that are included in my AMI registration tools. So I can't make those public right now. The process itself can be replicated relatively easily, though:

    1. Build your VM image using the build-backend and builder-image repos above. PXE is what I use, but you could just as easily make it into an ISO or something. If you intend to do an S3-backed AMI, you will need to make the image no larger than 10GB (I use 8GB).
    2. Trim the image down (I do a 'mount -o loop,discard' on the image, then 'fstrim' the mount point, making the image into a sparse file).
    3. Tarball the image (tar cSzf, S to preserve the sparseness).
    4. Upload the tarball to S3.
    5. In each region, launch an instance and attach an empty 8GB EBS volume to them.
    6. On each of those instances, download the tarball and extract with 'tar xSf'.
    7. Use 'ddpt', an enhanced dd which pays attention to the sparseness of the image, to copy the raw image file into the EBS device. I use "ddpt if=<imagefile> of=/dev/xvdf bs=512 conv=sparse oflag=sparse,fsync". The sparseness aspect is important, because otherwise you're copying empty blocks onto the EBS device, which makes the snapshot take much longer, and is really just a waste of time. EBS volumes already read-as-zero, so there's no sense copying zero blocks.
    8. Detach the EBS volume and terminate the instances.
    9. Snapshot the EBS volume.
    10. Delete the volume (not needed now).
    11. Use ec2-register to create an AMI from the snapshotted volume.

  • 2013-03-22

    I've started creating AMIs which have CUDA preinstalled. These are for the cg1.4xlarge instance type.

  • 2013-02-05

    Nothing too exciting lately. Today's release has Linux 3.7.6.

  • 2012-11-22

    New AMI releases, now with cloud-init. Thanks to Jeremy D for contributing his time and effort to making cloud-init work well on Arch Linux.

  • 2012-11-12

    Released new AMIs, primarily for the new AWS region in Sydney, Australia (ap-southeast-2).

  • 2012-11-08

    Today's AMIs are released. Nothing too fancy in this build: just updated packages, including linux-ec2 3.6.6-1.

  • 2012-10-21

    I've added a new linux-ec2 package which contains a patched v3.6.2 kernel. There are a few major differences between this kernel and the Arch Linux stock kernel:

    • Hangs on Xen fixed (patches from 3.6.3 stable-queue).
    • CONFIG_PREEMPT_VOLUNTARY instead of CONFIG_PREEMPT, this will allow for better scheduling as a domU.
    • CONFIG_HZ=100 instead of CONFIG_HZ=300, this allows for better performance on many-CPU instances, as there are fewer timer interrupts to preempt other tasks.
    • Many drivers removed, particularly those that didn't make sense for running in an EC2 instance. I've left drivers for my own hardware so I can experiment with it as a dom0 kernel as well. The kernel size is roughly half the stock Arch Linux kernel due to the stripped drivers.

    I am also building new AMIs right now, and am beating the i386 AMIs into working order. Once done I'll publish the next release (which should be 2012.10.21). Once it's available, it will show in the tables above.

  • 2012-10-16

    Do not upgrade HVM instance kernels to anything between 3.6.0 and 3.6.2 inclusive. You must wait for 3.6.3 or else your instance will not boot. We're currently waiting on this patch to be integrated into the mainline stable tree. This is also why I am probably not doing an AMI release this week, as the HVM AMIs would be totally broken.

    I've also taken a look at building i386 (well, i686) AMIs. I'm not really sure that it's worth the effort. Nobody really uses 32-bit AMIs anymore, and we'd need to fork the kernel just to make it happen. For now, i686 is on ice.

release notes

These AMIs are as close to a "vanilla" install as I can make them without making them functionally impaired on EC2. But here's the complete list of differences between the EC2 builds and a stock install:

  • High performance kernel specifically for EC2, including paravirtualization support on i386 and x86_64 AMIs, and more Xen-friendly process scheduling.

  • Kernel modules included in initrd, some of which are relevant outside of EC2 contexts (e.g. if you want to run the image in a non-EC2 environment such as KVM or Hyper-V):

    • KVM: virtio virtio-blk virtio-net virtio-pci virtio-ring
    • Xen: xen-blkfront xen-netfront xen-pcifront xen-privcmd
    • Hyper-V: hv_storvsc hv_balloon hv_vmbus hv_utils hv_netvsc
    • IPMI (e.g. EC2 reboot request): button ipmi-msghandler ipmi-poweroff
    • EC2 enhanced networking SR-IOV driver: ixgbevf
  • Extra packages installed: audit, cloud-init, ec2-keyring, ec2-pacman-mirrors, irqbalance, lrzip, openssh, rng-tools, rsync, systemd-sysvcompat

  • Added an extra package source for ec2-specific packages. The repository currently contains numerous packages useful on EC2. You can view the list of packages by doing 'pacman -Sy; pacman -Ss | grep ^ec2'

  • Additional services enabled at boot: rngd, sshd, cloud-init, irqbalance, auditd, systemd-networkd, systemd-timesyncd, systemd-resolved

  • User's public key is pulled from the EC2 instance metadata service at startup, and added to /root/.ssh/authorized_keys

  • SSH configured with 'PasswordAuthentication no', enforcing public key authentication

  • pacman loads (and automatically lsigns) the 'archlinux' and 'ec2' keyrings on the first boot (the latter keyring contains my public key used for package signing in the ec2 repo).

  • pacman mirror list is automatically selected at boot based on a list I created (based on rankmirrors run on instances in each region). These lists are provided by the package ec2-pacman-mirrors, which is in the ec2 repo.

  • dhclient is used instead of dhcpcd for robustness reasons. I found that dhcpcd gave up too quickly if it tried to do a DHCPREQUEST when the vif wasn't completely up, making the EC2 instance inaccessible.

  • dhclient is configured to retry forever, and request the following dhcp options: subnet-mask, broadcast-address, time-offset, routers, domain-name, domain-name-servers, host-name, interface-mtu, fqdn

  • /usr/bin/pinentry is symlinked to /usr/bin/pinentry-curses instead of the default pinentry-gtk, since gtk isn't available in this install and the primary access method is SSH.