Small Docker images with multi-stage builds

Image size sounds like a vanity metric until you have forty services, a registry bill and a deploy that spends most of its time pulling layers. Smaller images also mean fewer packages for the security scanner to complain about and a smaller attack surface if something does go wrong.

The single biggest win I know is the multi-stage build:

1. one stage has the full toolchain and builds the artifact;
2. the final stage copies only the artifact into a minimal base.

The compiler, the package cache and your source code never reach production.

A Go service#

Go is the easy case because it can produce a static binary.

# syntax=docker/dockerfile:1
FROM golang:1.23 AS build
WORKDIR /src

# Dependencies first: this layer is reused until go.mod or go.sum change.
COPY go.mod go.sum ./
RUN go mod download

COPY . .
RUN CGO_ENABLED=0 go build -trimpath -ldflags="-s -w" -o /out/app ./cmd/app

FROM gcr.io/distroless/static-debian12:nonroot
COPY --from=build /out/app /app
USER nonroot:nonroot
ENTRYPOINT ["/app"]

A few details carry most of the weight here:

• CGO_ENABLED=0 gives a fully static binary, so it runs on a base image with no libc at all.
• -trimpath strips local file system paths from the binary, which also makes builds more reproducible.
• -ldflags="-s -w" drops the symbol table and DWARF debug info. Stack traces still work; you lose the ability to attach a debugger to the production binary, which I’ve never needed.
• distroless/static is about 2 MB and has no shell and no package manager, just CA certificates, timezone data and a nonroot user.

The build image is several hundred megabytes. The final one is the base plus your binary, usually 10–25 MB for a typical HTTP service.

Cache mounts#

Ordering COPY instructions helps, but every change to go.mod still downloads all modules again, and every code change recompiles from scratch inside the build. BuildKit cache mounts fix both:

RUN --mount=type=cache,target=/go/pkg/mod \
    --mount=type=cache,target=/root/.cache/go-build \
    CGO_ENABLED=0 go build -trimpath -ldflags="-s -w" -o /out/app ./cmd/app

The cache directories persist between builds on the same builder but never end up in an image layer. On my laptop this took incremental builds from about a minute to a few seconds.

A Python service#

Python doesn’t compile to a single file, so the trick is to build a virtualenv in one stage and copy it over:

# syntax=docker/dockerfile:1
FROM python:3.12-slim AS build
RUN python -m venv /venv
ENV PATH=/venv/bin:$PATH
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

FROM python:3.12-slim
COPY --from=build /venv /venv
ENV PATH=/venv/bin:$PATH \
    PYTHONDONTWRITEBYTECODE=1 \
    PYTHONUNBUFFERED=1
WORKDIR /app
COPY src/ ./src/
USER 1000:1000
CMD ["python", "-m", "src.main"]

The point of the build stage is that anything you install to compile wheels (gcc, libpq-dev and friends) stays there.

On a project with psycopg, numpy and a dozen smaller packages this got us from just over 1 GB (the full python:3.12 image with build tools left in) to around 250 MB. Not tiny, but most of what’s left is the libraries we actually use.

The .dockerignore you forgot#

COPY . . copies the whole build context, and without a .dockerignore that includes .git, local virtualenvs, test fixtures and whatever node_modules is lying around. Even when those files end up in a build stage only, they bust the cache on every commit and slow down the context upload. Mine usually starts like this:

.git
.venv
node_modules
**/__pycache__
*.log
dist/
.env*

That last line is not about size at all: it keeps local secrets out of the image.

Debugging an image without a shell#

The first time someone needs to poke around inside a distroless container, they’ll be annoyed that docker exec -it app sh fails. Two things help:

# Build and run only the build stage, which has a shell and all tools.
docker build --target build -t app:debug .
docker run --rm -it app:debug bash

# Inspect what each layer of the final image adds.
docker history app:latest

Is it worth it?#

For compiled languages, absolutely: it’s a ten-line change and the result is better in every way. For Python, Node and Ruby the gains are smaller, but keeping compilers out of the runtime image is worth doing even if the size doesn’t move much. The scanner reports get shorter, and that’s the part my security team actually notices.