Upgrading docker.io when it complains about AUFS



So I got this error recently…

root@persimmon ~ # apt install docker.io
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Suggested packages:
  aufs-tools cgroupfs-mount | cgroup-lite debootstrap docker-doc rinse zfs-fuse | zfsutils
The following packages will be upgraded:
1 to upgrade, 0 to newly install, 0 to remove and 103 not to upgrade.
1 not fully installed or removed.
Need to get 0 B/36.9 MB of archives.
After this operation, 102 kB of additional disk space will be used.
Preconfiguring packages ...
(Reading database ... 182205 files and directories currently installed.)
Preparing to unpack .../docker.io_20.10.7-0ubuntu5~18.04.3_amd64.deb ...
The aufs storage-driver is no longer supported.
Please ensure that none of your containers are
using the aufs storage driver, remove the directory
/var/lib/docker/aufs and try again.
dpkg: error processing archive /var/cache/apt/archives/docker.io_20.10.7-0ubuntu5~18.04.3_amd64.deb (--unpack):
 new docker.io package pre-installation script subprocess returned error exit status 1
Errors were encountered while processing:
E: Sub-process /usr/bin/dpkg returned an error code (1)

Evidently, docker on Ubuntu 18.04 upgraded to a version where AUFS support was dropped. I’m a little surprised at this because Ubuntu LTS usually doesn’t upgrade packages in ways that break compatibility.

So how does one proceed here? You can just delete the directory you’re instructed to, but that might be a little messy, so here’s a small document on how to do it cleanly.

Before you start

Before you upgrade docker through this version, you should make sure you know where all of your docker containers come from. Run docker ps, and see if you know how to restart everything. If you don’t, then you probably aren’t ready to upgrade here. If you do, then you will need to make a mental note of what needs starting once everything is upgraded.

Here is how the process will go:

If you have containers that can’t just be deleted because they’ll lose state, then I’ll give a little warning and say that docker isn’t designed for that—it’s supposed to be stateless—but you can rescue a modified container by creating an image from it using the docker commit command, then later creating it from that image instead of the original.

Saving everything

Making a note of image tags

Run docker images. You might get output like this:

REPOSITORY            TAG       IMAGE ID       CREATED       SIZE
<none>                <none>    741887401ee4   3 years ago   2.17GB
gentoo/stage3-amd64   latest    ff47b7297bd7   3 years ago   963MB
ubuntu                latest    452a96d81c30   3 years ago   79.6MB
archlinux_build       latest    25ad31135fc5   4 years ago   940MB
base/archlinux        latest    c6ae11e0cb4d   4 years ago   652MB
<none>                <none>    97434d46f197   5 years ago   188MB

Ignore all the <none>s, and write down the others. You only need the first 3 columns:

gentoo/stage3-amd64   latest    ff47b7297bd7
ubuntu                latest    452a96d81c30
archlinux_build       latest    25ad31135fc5
base/archlinux        latest    c6ae11e0cb4d

Make sure this is saved in some txt file somewhere.

Dumping images

Save all your docker images to a tar file with this nifty command:

docker save $(docker images -q) > images.tar

That will take a reasonably long time to run and won’t give any output. You might need a lot of disk space too. If that’s a problem you might want to pipe it through a compression tool or something but that’s left as an exercise to the reader.

How do I check that all the images are saved correctly?

Beats me 🤷‍♀️ Check the return code of the previous command (using echo $?) and make sure it says 0? Standard stuff, probably not ideal but better than nothing.

Deleting things

From here on out, things aren’t really recoverable if you mess up. Keep backups if you’re worried. Full-system ones, including the docker binaries.

Removing docker containers

First stop all docker containers:

docker stop $(docker ps -q)

If any fail, you can use docker kill instead of docker stop with the command above, but it’s probably a bad omen so I’d bail and investigate at that point.

After stopping them, you can remove them:

docker rm $(docker ps -a -q)

(Here the -a instructs it to list all containers instead of just the actively running ones.)

Removing docker images

It’s a very similar process. Once the above steps have been completed, you should be able to just…

docker rmi $(docker images -a -q)

This might complain. When doesn’t it. If it complains, stick an -f in there and call it a day.1

docker rmi -f $(docker images -a -q)

If you get “no such image” errors, that’s fine, ignore them.

Stop docker

The next step might confuse docker a little, so let’s put it to sleep first:

sudo systemctl stop docker

Remove the AUFS directory

As instructed by the output of apt when installing docker.io, you will need to remove the /var/lib/docker/aufs directory.

sudo rm -r /var/lib/docker/aufs

Upgrade docker

But first! Check you don’t have AUFS configured

sudo cat /etc/docker/daemon.json

That shouldn’t mention aufs. If it does, it will probably be something like "storage-driver": "aufs", in which case you should open it up in a text editor and change aufs to overlay2. Save and exit.

If you got “file not found” or something, that’s Good Actually (means all config is default), so let’s go on.

Do the actual upgrade

sudo apt update
sudo apt install docker.io

Yup, install, not upgrade.2

It might ask you if it’s fine to restart docker. Doesn’t really matter what you answer to that, because…

Start docker

…we make sure it’s (re)started anyway.

sudo systemctl restart docker

Restore everything

Restore images

set -o noclobber
docker load < images.tar

I’ve taken the liberty of setting noclobber in the shell there. That way if you mistype the redirection in a way that wipes images.tar, it will fail instead of wiping it.

This might take a similarly long time as it did to save, perhaps even longer. But at least it will be a little more musical about it.

Re-tag images

Remember all the image tags that we saved to some text file somewhere and definitely didn’t just skip?

You will want to rearrange these into commands.3 Move the third column into the front, and put a colon between the other two, then prefix with docker tag like so:

docker tag ff47b7297bd7 gentoo/stage3-amd64:latest
docker tag 452a96d81c30 ubuntu:latest    
docker tag 25ad31135fc5 archlinux_build:latest    
docker tag c6ae11e0cb4d base/archlinux:latest    

Run each of those commands you have now constructed. Yours will not be the same as mine so don’t copy-paste mine in the hopes that they’ll work. After this, your docker images command should show all the tag names instead of just anonymous IDs.

Recreate containers

Okay, everything is 90% done now. All you need to do is recreate and restart your containers. Knowing how to do this was step 1 of the post, and this is again different for everyone, so go ahead and do this.


That’s all! The containers are now running on a newer version of docker without AUFS, and you should hopefully not run into any issues like this again for the foreseeable future.

Updated on 2021-12-21: formatting