Software Installations: Create Multi-arch Docker Images using Docker BuildX CLI plugin

Friday, May 8, 2020

Create Multi-arch Docker Images using Docker BuildX CLI plugin

Software:

MacOS 10.15.4 64-bit (Catalina)
Ubuntu Server 19.04 64-bit (eoan)
Raspbian GNU/Linux 10 32-bit (buster)
Docker/Buildx v0.3.1-tp-docker

Hardware:

MacBook:

MacOS 64-bit OS
Docker 19.03.8

Raspberry Pi 1, 2, 3 or 4:

Raspbian 10 32-bit OS
Docker 18.09.1

Raspberry Pi 3 or 4:

Ubuntu Server 64-bit OS
Docker 19.03.6

Goals:

Verify that Docker images are platform dependent (single-arch).
Learn how to build multi-arch docker images that can be used on several platforms (e.g. amd64, arm64, arm).

Prerequisites:

An externally-accessible insecure Docker Registry running on the MacOS machine. See how to configure it here
Configure Docker to use experimental features
Configure Docker to work with insecure-registry

Build a simple Docker Image:

Let's start creating a simple Dockerfile to build an image:

mkdir ~/docker-test
cd ~/docker-test
nano ./Dockerfile

FROM ubuntu:18.04
CMD ["sleep", "infinity"]

Build an image on a MacBook and tag as AMD64:

docker login 192.168.1.107:5000
docker build -t 192.168.1.107:5000/docker-test:amd64 .

Push the image to the private docker registry:

docker push 192.168.1.107:5000/docker-test:amd64

Try to run a container based on this image:

Machine: MacBook

docker run --entrypoint bash -it 192.168.1.107:5000/docker-test:amd64

root@d465832164be:/# uname -m
x86_64

Machine: Raspberry Pi running Ubuntu 64-bit OS

docker run --entrypoint bash -it 192.168.1.107:5000/docker-test:amd64

standard_init_linux.go:211: exec user process caused "exec format error"

Machine: Raspberry Pi running Raspbian 32-bit OS

docker run --entrypoint bash -it 192.168.1.107:5000/docker-test:amd64

standard_init_linux.go:207: exec user process caused "exec format error"

Build an image on Raspberry Pi running Raspbian 32-bit OS and tag as ARM32

Copy the Dockerfile to a folder on Raspberry Pi 32-bit OS machine and `cd` to it:

docker login 192.168.1.107:5000
docker build -t 192.168.1.107:5000/docker-test:arm32 .

Push the image to the private docker registry:

docker push 192.168.1.107:5000/docker-test:arm32

Try to run a container based on this image:

Machine: MacBook

docker run --entrypoint bash -it 192.168.1.107:5000/docker-test:arm32

root@ead4869f18b9:/# uname -m
armv7l

Machine: Raspberry Pi running Ubuntu 64-bit OS

docker run --entrypoint bash -it 192.168.1.107:5000/docker-test:arm32

root@95b80ff7d56f:/# uname -m
aarch64

Machine: Raspberry Pi running Raspbian 32-bit OS

docker run --entrypoint bash -it 192.168.1.107:5000/docker-test:arm32

root@d8bbc13939e5:/# uname -m
armv7l

Build an image on Raspberry Pi running Ubuntu 64-bit OS and tag as ARM64:

Copy the Dockerfile to a folder on Raspberry Pi 32-bit OS machine and `cd` to it:

docker login 192.168.1.107:5000
docker build -t 192.168.1.107:5000/docker-test:arm64 .

Push the image to the private docker registry:

docker push 192.168.1.107:5000/docker-test:arm64

Try to run a container based on this image:

Machine: MacBook

docker run --entrypoint bash -it 192.168.1.107:5000/docker-test:arm64

root@feb84b4dfc5c:/# uname -m
aarch64

Machine: Raspberry Pi running Ubuntu 64-bit OS

docker run --entrypoint bash -it 192.168.1.107:5000/docker-test:arm64

root@c9e0aa8eb98e:/# uname -m
aarch64

Machine: Raspberry Pi running Raspbian 32-bit OS

docker run --entrypoint bash -it 192.168.1.107:5000/docker-test:arm64

standard_init_linux.go:207: exec user process caused "exec format error"

Listing Docker images:

docker images | grep 192.168.1.107

192.168.1.107:5000/docker-test arm32 51c74aa10614 46.7MB
192.168.1.107:5000/docker-test arm64 c6dd32c882f6 57.7MB
192.168.1.107:5000/docker-test amd64 32458c475b0e 64.2MB

docker image inspect 192.168.1.107:5000/docker-test:arm32 | grep Architecture

"Architecture": "arm",

docker image inspect 192.168.1.107:5000/docker-test:arm64 | grep Architecture

"Architecture": "arm64",

docker image inspect 192.168.1.107:5000/docker-test:amd64 | grep Architecture

"Architecture": "amd64",

Conclusions:

The images built with `docker build` command are platform dependent.
An image built on ARM32 platform CAN be used on ARM64 platform.
An image built on ARM64 platform can NOT be used on ARM32 platform.
The Docker Desktop (for MacOS and Windows) has QEMU emulation and can run many platform images, regardless of the platform the image was built for.

QEMU emulation for the arm/v6, arm/v7 and arm64 Docker images

Build Multi-arch simple Docker Images:

Let's start creating a simple Dockerfile to build an image:

mkdir ~/docker-test-multiarch
cd ~/docker-test-multiarch
nano ./Dockerfile

FROM ubuntu:18.04
ARG TARGETPLATFORM
ARG BUILDPLATFORM
RUN echo "I am running on $BUILDPLATFORM, building for $TARGETPLATFORM"
CMD ["sleep", "infinity"]

Build multi-architecture images on a MacBook or Linux:

cd ~/docker-test-multiarch
Log in on the registry server:

docker login -u <registry-user> [registry_url]

enter the registry password

Create a new instance of an isolated builder:

docker buildx create --name multiarch-builder --platform linux/amd64,linux/arm64

Switches the current builder instance. Build commands invoked after this command will run on a specified builder.

docker buildx use multiarch-builder

[Optional] docker buildx ls
[Optional] docker buildx inspect multiarch-builder
Build the images for the desired platforms (architectures):

docker buildx build -t <registry-url>/<image-name>:<tag> [-f Dockerfile] --platform linux/amd64,linux/arm64 --push .

Cleanup the environment:

docker buildx use default
docker buildx stop multiarch-builder
docker buildx rm multiarch-builder

References: