Multi-stage Docker builds are your friend
For my first real post of my new blog, let’s talk about multi-stage docker builds. This blog is built with the aid of just such a build. A multi-stage docker build gives you the ability to build multiple containers. In the case of my build, it helps me build a single end-product that’s uncluttered by extra build configuration and tooling.
Docker holds a place in the modern development pipeline for deployed
applications similar to the role the classic Makefile has in building packaged
applications. (There’s a reason the
Makefile use a similar
nomenclature after all.) However, when building a deployed application, you
often need to perform build tasks that create clutter. For example, if you’re
building a C program, you’ll like need to install the build tools (
sudo apt-get install build-essential) and then when you run
make you get all those
intermediate .o files needed for linking, and maybe you need to install Perl or
Python to help run some of the glue code, etc.
Ideally, to save space, improve security, and just generally avoid having extra junk laying around, you want to avoid having all that flotsam and jetsam laying about in your container images. You could try to be extra diligent and uninstall those things and delete your files, but you probably won’t and there’s no guarantee you’ll really be able to get things pristine again. Furthermore, due to the multi-layered nature of a docker container image, those files you delete are still sort of there, they’ve just been hidden.
One way to rescue yourself from these problems is to use a multi-stage build in
Dockerfile. A multi-stage container helps clean-up the clutter without
having to perform rm commands or
apt-get remove. It also prevents carrying
around deleted files in hidden layers because your final build can simply omit
those layers when you use a multi-stage build.
How to multi-stage build?
A multi-stage build is really easy to do. First, make sure you have the latest
version of Docker installed because this is a newer feature. (Also, security
updates are thing. Why would you run an older version of Docker!?) Second, you
need to have multiple
FROM commands in your
what amounts to a new container image:
FROM debian:latest AS builder RUN apt-get install git build-essential cmake COPY /c-src /scratch WORKDIR /scratch RUN make FROM rakudo-star:latest AS release COPY --from=builder /scratch/cool-program /usr/bin/cool-program COPY /p6-src /app ENTRYPOINT [ "perl6", "/app/bin/cooler-program", "--cool-program-path=/usr/bin/cool-program" ]
This Dockerfile will now build two separate container images. The second of which will
contain a file built in the first. All the extra source code and junk added to
cool-program is only in the build container and left ouf of the
final. This leaves your
cooler-program completely free of all that unnecessary
If you build this and push the release container to Docker hub, only the build system is going to have the parts of the image for builder. You can, if you want, push the other images in the multi-stage build, but for this example, I probably wouldn’t. Only the release is important.
This Web Site
I am write this because I wrote just such a
Dockerfile to build this web site,
which is built from a three-stage Dockerfile.
- Stage 1: Build MultiMarkdown from source.
- Stage 2: Install the Perl 6 code for the static site generator and all dependencies.
- Stage 3: Copy the built bits to the final result for deployment.
This demonstrates how to cleanly build a Perl 6 installation without keeping the original source around.
Regarding Perl 6 Builds
If you are doing this with Perl 6, I the following knowledge may save you some
time: When you use
zef to do your build on a
rakudo-star base container, all
the build files go into the directory named
/usr/share/perl6/site. You can
safely copy that from one Rakudo* to another Rakudo* and you will have
everything you need.
P.S. Here’s the source for this
Dockerfile as of this writing. The above
example is shorter, but I haven’t actually run that one. This one should be
working because I copied it straight from the project into here.
FROM debian:latest AS multimarkdown-builder RUN apt-get update -y RUN apt-get install -y git build-essential cmake RUN mkdir /scratch WORKDIR /scratch RUN git clone --recursive https://github.com/fletcher/MultiMarkdown-6.git multimarkdown \ && cd multimarkdown \ && make release \ && cd build \ && make FROM rakudo-star:latest AS zostay-builder RUN zef update COPY . /app WORKDIR /app RUN zef install . FROM rakudo-star:latest AS zostay COPY --from=multimarkdown-builder \ /scratch/multimarkdown/build/multimarkdown \ /usr/bin/multimarkdown COPY --from=zostay-builder \ /usr/share/perl6/site \ /usr/share/perl6/site VOLUME /src VOLUME /dst ENTRYPOINT ["/usr/share/perl6/site/bin/zostay"] CMD ["build-loop", "/src", "/dst"]