Programming Environment

Author

Jon Reades

Published

September 7, 2023

Over the years, we have experimented with a range of approaches to setting you up with a programming environment, and we have come to the conclusion that Docker1 is the most robust way to ensure a consistent experience. This guarantees that all students end up with the same versions of each library, that difficult-to-diagnose hardware/OS issues are minimised, and that running/recovery is the most straightforward.

Base Utilities

Before beginning any of the steps below, make sure you have installed the Base Utilities for your Operating System and checked the other hardware and software requirements.

Downloading

What Kind of Computer?

Apple Intel or Silicon/M-chip?

Apple has stopped using Intel computer chips and now uses its own ‘Silicon’ (aka ‘M’) chips instead. Confusingly, these are often also referred to by their technical specification as AMD64 (Intel) and ARM64 (Apple), respectively. To find out which you have, click on the Apple icon () and select About this Mac. Look at the Processor section and you should see which type of computer you have. If you see the word Intel then you have an Intel computer, if you see M1, M2, M3, etc. then you have an Apple Silicon computer.

Windows 32- or 64-bit?

It is highly unlikely that you have a 32-bit machine, so if in doubt download the 64-bit version. It is also unlikely that you have an ARM device if you have an off-the-shelf Windows computer.

While you’re waiting for the packages below to finish downloading, you can also have a look at the information about getting set up on GitHub…

Please download all of the Essential packages for your type of computer (Apple Intel, Apple Silicon, Windows). The rest are listed for your convenience and you can install them as you see fit.

Next Steps

R and R-Studio can be installed fairly easily by double-clicking on the downloaded files and following the instructions. Docker is a bit more complex, so please follow the instructions below.

Installing Docker

Essential Topic

This next section is essential to running the sds2024 environment.

Docker is a complex application doing very complex things. It’s not surprising that it can be a bit of a pain to install. But once installed, it’s a very powerful platform for ‘doing (spatial) data science’ that’s widely used in industry and, increasingly, academia.

One way to think of it as a ‘library’ of ready-made virtual computers that you can copy and use free-of-charge. If you’d like to know more about what Docker is and how it works, you can read more in the Understanding Docker section.

Windows Users

Please ensure that you have installed WSL2 before installing Docker! If you cannot install WSL2 then please have a look at the ‘dealing with errors’ section.

After you’ve downloaded Docker, you need to:

  1. Install it – usually this will mean opening the image and either dragging it your Application folder (Mac) or running the installer (Windows)
  2. Start it up – double-click the Docker icon in your Applications folder to start Docker running.
  3. Finish setup – once Docker is finished starting up, you should see the login screen below. You do not need to create an account (notice Continue without signing in)
  4. On all the subsequent questions you can Skip (upper-right corner) answering as well.

Docker trying to trick you into creating an account

You must finish setting up before proceeding to the next step. You’ll know that you’re ready to move on when you see the ‘Docker Desktop’ window appear listing ‘downloaded images’ and ‘running containers’:

Docker Desktop

If you didn’t see this then you will need to have a look at the ‘dealing with errors’ section.

Testing Docker

To test if Docker is installed correctly, you will need either the Terminal (macOS) or the Power Shell (Windows).

Copy Code to Clipboard

Whenever you see a ‘code block’ below, you will also see a ‘clipboard’ icon (Copy to Clipboard Icon) in the right. Click that, and the code will be copied to your computer’s ‘clipboard’ so that you can then paste it into the Terminal or Power Shell. That will save you a lot of time and effort.

Docker has provided a simple way to test if your installation is working correctly. You can run the following command in the Terminal or Power Shell:

docker run hello-world

This should output something like:

Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
478afc919002: Pull complete
Digest: sha256:91fb4b041da273d5a3273b6d587d62d518300a6ad268b28628f74997b93171b2
Status: Downloaded newer image for hello-world:latest

Hello from Docker!
This message shows that your installation appears to be working correctly.

To generate this message, Docker took the following steps:
 1. The Docker client contacted the Docker daemon.
 2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
    (arm64v8)
 3. The Docker daemon created a new container from that image which runs the
    executable that produces the output you are currently reading.
 4. The Docker daemon streamed that output to the Docker client, which sent it
    to your terminal.

To try something more ambitious, you can run an Ubuntu container with:
 $ docker run -it ubuntu bash

Share images, automate workflows, and more with a free Docker ID:
 https://hub.docker.com/

For more examples and ideas, visit:
 https://docs.docker.com/get-started/

If you see this message, then Docker is installed correctly and you can move on to the next step. If you see an error message, then you will need to have a look at the ‘dealing with errors’ section. Notice how there are several things happening:

  1. Unable to find image... locally – this is because the hello-world image is not (yet) stored on your computer.
  2. latest: Pulling from library/hello-world – this is Docker downloading the hello-world image from the internet.
  3. Pull complete – this is Docker telling you that it has finished downloading the image.
  4. Hello from Docker! – this is the hello-world image running and telling you that Docker is working correctly.

There’s obviously a lot more to that message, but that’s the basic idea.

Installing sds2024

We now need to download the sds2024 image that we use for teaching Foundations and Quantitative Methods. The image is the ‘template’ for running virtual machines (i.e. computers) on our ‘host’ computer and it comes complete with all of the Python libraries and other tools that you’ll need to complete the module (and a good deal more besides!). Installing the image will take a while as it’s quite large (2-5GB) and so will depend on the speed of your internet connection.

If your Apple computer has an Intel chipset (see:  -> About this Mac -> Processor) or is running Windows:

docker pull jreades/sds:2024-intel

If your Apple computer has an M1 or M2 ‘Silicon’ chipset (see:  -> About this Mac -> Processor):

docker pull jreades/sds:2024-silicon

Using jreades/sds:2024

Now that you have the image downloaded, you can start an sds2024 container by copying the following command into the Terminal or Power Shell (this does not work in the Command Prompt/cmd):

docker run --rm -d --name sds2024 -p 8888:8888 \
   -v "$(pwd):/home/jovyan/work" \
  jreades/sds:2024-intel start.sh jupyter lab \
  --LabApp.password='' --ServerApp.password='' --NotebookApp.token=''
docker run --rm -d --name sds2024 -p 8888:8888 \
   -v "$(pwd):/home/jovyan/work" \
  jreades/sds:2024-silicon start.sh jupyter lab \
  --LabApp.password='' --ServerApp.password='' --NotebookApp.token=''
docker run --rm -d --name sds2024 -p 8888:8888 -v "$(pwd):/home/jovyan/work" jreades/sds:2024-intel start.sh jupyter lab --LabApp.password='' --ServerApp.password='' --NotebookApp.token=''

Success!

However, most of you should now be able to connect to the virtual machine by pointing your browser at: localhost:8888 where you should see something like this:

Jupyter Lab Success

When Things Go Wrong

If you encounter any problems with the installation, please have a look at the Dealing with Docker Problems page.

Footnotes

  1. And Podman, which is basically the same thing.↩︎