For my first real post of my new blog, let’s talk about multi-stage docker builds. This blog is built with the aid of just such a build. A multi-stage docker build gives you the ability to build multiple containers. In the case of my build, it helps me build a single end-product that’s uncluttered by extra build configuration and tooling.

Why multi-stage?

Docker holds a place in the modern development pipeline for deployed applications similar to the role the classic Makefile has in building packaged applications. (There’s a reason the Dockerfile and Makefile use a similar nomenclature after all.) However, when building a deployed application, you often need to perform build tasks that create clutter. For example, if you’re building a C program, you’ll like need to install the build tools (sudo apt-get install build-essential) and then when you run make you get all those intermediate .o files needed for linking, and maybe you need to install Perl or Python to help run some of the glue code, etc.

Ideally, to save space, improve security, and just generally avoid having extra junk laying around, you want to avoid having all that flotsam and jetsam laying about in your container images. You could try to be extra diligent and uninstall those things and delete your files, but you probably won’t and there’s no guarantee you’ll really be able to get things pristine again. Furthermore, due to the multi-layered nature of a docker container image, those files you delete are still sort of there, they’ve just been hidden.

One way to rescue yourself from these problems is to use a multi-stage build in your Dockerfile. A multi-stage container helps clean-up the clutter without having to perform rm commands or apt-get remove. It also prevents carrying around deleted files in hidden layers because your final build can simply omit those layers when you use a multi-stage build.

How to multi-stage build?

A multi-stage build is really easy to do. First, make sure you have the latest version of Docker installed because this is a newer feature. (Also, security updates are thing. Why would you run an older version of Docker!?) Second, you need to have multiple FROM commands in your Dockerfile. Each FROM starts what amounts to a new container image:

FROM debian:latest AS builder

RUN apt-get install git build-essential cmake
COPY /c-src /scratch

WORKDIR /scratch
RUN make

FROM rakudo-star:latest AS release 

COPY --from=builder /scratch/cool-program /usr/bin/cool-program

COPY /p6-src /app
ENTRYPOINT [
    "perl6", 
    "/app/bin/cooler-program", 
    "--cool-program-path=/usr/bin/cool-program"
]

This Dockerfile will now build two separate container images. The second of which will contain a file built in the first. All the extra source code and junk added to build the cool-program is only in the build container and left ouf of the final. This leaves your cooler-program completely free of all that unnecessary clutter.

If you build this and push the release container to Docker hub, only the build system is going to have the parts of the image for builder. You can, if you want, push the other images in the multi-stage build, but for this example, I probably wouldn’t. Only the release is important.

This Web Site

I am write this because I wrote just such a Dockerfile to build this web site, which is built from a three-stage Dockerfile.

1. Stage 1: Build MultiMarkdown from source.
2. Stage 2: Install the Perl 6 code for the static site generator and all dependencies.
3. Stage 3: Copy the built bits to the final result for deployment.

This demonstrates how to cleanly build a Perl 6 installation without keeping the original source around.

Regarding Perl 6 Builds

If you are doing this with Perl 6, I the following knowledge may save you some time: When you use zef to do your build on a rakudo-star base container, all the build files go into the directory named /usr/share/perl6/site. You can safely copy that from one Rakudo* to another Rakudo* and you will have everything you need.

Cheers.

P.S. Here’s the source for this Dockerfile as of this writing. The above example is shorter, but I haven’t actually run that one. This one should be working because I copied it straight from the project into here.

FROM debian:latest AS multimarkdown-builder

RUN apt-get update -y
RUN apt-get install -y git build-essential cmake

RUN mkdir /scratch
WORKDIR /scratch

RUN git clone --recursive https://github.com/fletcher/MultiMarkdown-6.git multimarkdown \
    && cd multimarkdown \
    && make release \
    && cd build \
    && make

FROM rakudo-star:latest AS zostay-builder

RUN zef update

COPY . /app

WORKDIR /app

RUN zef install .

FROM rakudo-star:latest AS zostay

COPY --from=multimarkdown-builder \
    /scratch/multimarkdown/build/multimarkdown \
    /usr/bin/multimarkdown

COPY --from=zostay-builder \
    /usr/share/perl6/site \
    /usr/share/perl6/site 

VOLUME /src
VOLUME /dst

ENTRYPOINT ["/usr/share/perl6/site/bin/zostay"]
CMD ["build-loop", "/src", "/dst"]