Dealing with Docker Problems

There are two types of problems that you’re likely to encounter when trying to use Docker:

1. Errors installing Docker
2. Errors running Docker

The vast majority of people encountering these issues will be on Windows machines.

Errors Installing Docker

Unsupported Version

The only problem that is common to both Windows and macOS machines is the ‘unsupported version’ (or similar wording) error. This happens when you are running an older version of the Operating System that is no longer supported by Docker’s most recent installer.

If you are not able to update your computer’s operating system to a more recent release, then your only option is to use an older version of Docker. You can find older versions of Docker to install, though for very old machines this may still not resolve the ‘unsupported version’ problem.

Problems with WSL2

If you have problems that mention WSL/WSL2 (this could be errors installing or updating this system) and you are sure that you’ve installed and updated WSL, then you may need to try using ‘Hyper-V’ instead. This is a ‘second-best’ workaround that has worked for some people.

Enabling Hyper-V

Microsoft has instructions here on how to enable Hyper-V on your Windows machine. You will need to restart your computer after enabling Hyper-V. These instructions may also help.

You will then need to look at the Docker settings and change the Use the WSL2 based engine to Use the Hyper-V based engine.

Problems with Admin Rights

If you are unable to install Docker on your computer (e.g. because you are an apprentice with a corporate laptop), you may wish to try using Podman instead. Podman is runs in ‘user space’, which means that it does not need admin rights to run. You can find out more about Podman here. Our limited testing suggests that it works well with the SDS environment.

You will need to:

1. Download and install the Podman CLI and Desktop from here.
2. Run the following command in the PowerShell or Terminal:
   podman machine init
podman machine start
3. And then run this command in PowerShell or Terminal to download the SDS image with podman pull jreades/sds:2024-intel (or jreades/sds:2024-silicon for M-chip Macs)

Errors Running Docker

If you have managed to install Docker (or Podman, if it came to that) and have managed to ‘pull’ a disk image then the last remaining obstacle to running Docker is, typically, running out of disk space.

Dealing with Disk Space

Some Windows PCs (especially those sold with Windows10) have two drives (C and D). The C drive may fill up very quickly once you start installing and running software or processing data. In that case you may want to store your Docker images on the D drive.

This is a complex process, and you may need assistance, so there is a Moving to the D Drive page to try to talk you through the process.

Other Disk Space Issues

If you don’t have separate C and D drives but still have very little space left on your hard drive then it’s worth remembering that Docker is storing all of the images and containers that you have downloaded and run. You can see how much space Docker is using by running the following command in the PowerShell or Terminal:

docker system df

Deleting an Image

The most ‘intuitive’ way is to delete each image by it’s unique ID:

docker ps -aq # Get list of running processes and work out container IDs to remove
docker rm -f <list of container IDs>
docker images # Get list of available images and work out image IDs to remove
docker rmi -f <list of image IDs>

Deleting by Filter

A more ‘direct’ approach that should be used with some care is to looking for the name of the image and delete that way:

docker ps -aqf "name=sds" --format="{{.Image}} {{.Names}} {{.ID}}" | grep "2024" | cut -d' ' -f3 | xargs docker rm -f
docker images --format="{{.Repository}} {{.Tag}} {{.ID}}" | grep "sds" | cut -d' ' -f3 | xargs docker rmi

The Last Resort

Warning

We only support Anaconda Python as a fallback for students who would otherwise be unable to complete the module because their computer does not support Docker. If you choose to use Anaconda (but could run Docker) then we will not be able to support you.

A very small number of students are unable to run Docker or Podman at all on computers running Windows 10 Home (in 23/24 there were none), in which case Anaconda Python can be used with the configuration that we provide. However, if your machine runs Docker then you must use Docker: this isolates the programming environment from your computer, ensuring that nothing is clobbered by accident, and guaranteeing that you are working with the same version of every Python library as the rest of the class (and the versions for which the practicals are tested).

Anaconda is only supported as a last resort.

However you are always free to install Anaconda Python and to use our YAML configuration script to install the SDS environment, but you should do this in your own time: in previous years students have encountered difficult-to-diagnose bugs in their code (and lost marks in the Assessments!) because they had installed an older or more recent version of a Python library than the one configured and tested in the SDS environment.

We believe that the replication advantages of virtualisation outweigh the disbenefits in terms of performance. It also means that you will spend less time installing libraries and more time running code, which is where your attention should really be when you are familiarising yourself with the foundations of data science.

Eventually you will, of course, want to install and manage your own programming environment (possibly even by building and sharing Docker images!) but this can be left for later when you have developed an appreciation of how and when virtualisation is (or is not) an appropriate solution to your needs.