The image cache

One image per lab, tagged benchpress/<lab_id>:lab — what it costs on disk, how the weekly prewarm and sweep work, and what is safe to prune.

Lab images are the largest thing on a BenchPress host. This page says how they
are named, what keeps them, what removes them, and what you must never prune.

Who this is for. Whoever watches the disk.

Before you start. Read the numbers below before you order a rebuild. A
lab image on this host is between 5.5 GB and 19.7 GB, and building one
takes tens of minutes.

How an image is named

A lab's image is tagged benchpress/<lab_id>:lab. The tag comes from the
lab's identity, not from a hash of its recipe.

That has one useful consequence. The first Lab created from a catalog template
takes the template's own key as its lab_id, so a template's pre-warmed image
and the first unmodified Lab built from it resolve to the same tag with no
extra bookkeeping.

A Lab that then diverges from its template — edited apps, a different Frappe
version — needs its own build. Lab.validate() notices, and resets the lab's
status to Draft when the build spec no longer matches what was last built.

What is on this host

Twelve lab images, measured with docker images:

Those figures overlap heavily. Docker reports each image's full size, and
images built from the same base share layers. The whole image set on this host
occupies 54.74 GB, of which 19.12 GB is reclaimable.

Steps

1. Check what a build would cost before you order one.

   df -h /
   docker system df
   

   A deploy costs roughly 0.2 GB to 1 GB. A build costs the size of a new
   image. Do not start a build with a few gigabytes free.

2. Pre-warm the catalog so the first person to use a template does not pay
   for the build.

   bench --site <site> execute benchpress.image_cache.enqueue_prewarm_catalog
   

   It queues one job per template that has no cached image, so a single
   failing template cannot cancel the rest. It runs on queue-long, the only
   worker with a Docker socket. The golden runs in the same job, so a fresh
   host's catalog comes out golden rather than needing a second pass.

3. Sweep images nothing points at.

   bench --site <site> execute benchpress.image_cache.enqueue_sweep
   

   Both jobs also run weekly on their own.

4. Watch a build. Open Build history at /frontend/build-logs,
   reached from Labs. The screen is admin-only, and says so under its own
   heading.

   Screenshot: The Build history page in BenchPress, listing image builds across every lab in six columns — Lab, Image tag, Result, Last step, Duration and Started. The top row is Password Manager building benchpress/password-manager, Success, 26m 5s, 6 hours ago. One Frappe CRM row reads Failed after 44s, and several rows carry the image tag golden rather than a lab tag.

   Six columns: the lab, the image tag, the result, the last step reached, the
   duration and when it started. The slowest build on this host took 26
   minutes 5 seconds, and one CRM row failed after 44 seconds at
   Build started. Read the build history rather than the deploy log when a
   deploy fails for a lab that has never been built.

   A row tagged golden is not an image build. It is the pass that bakes
   the finished site's dump into an image that already exists. Those runs take
   5 to 11 minutes here, against 4 to 26 for a full build. See
   Golden images.

Verify

docker images --format '{{.Repository}}:{{.Tag}}\t{{.Size}}' | grep '^benchpress/'

Every tag ends in :lab. That suffix is what identifies an image BenchPress
built — the benchpress/ prefix alone would also catch a stray hand-tagged
image.

What the sweep keeps

The sweep removes cached images nothing needs. Four things count as needing
one:

That fourth case is the one worth understanding. A Building lab has no
image_tag yet, so its fresh image would look like an orphan and a sweep
could delete it out from under the deploy waiting for it. Hashing exactly
those labs' specs protects exactly those tags, so one lab stuck in Building
cannot disable the whole sweep.

Without the sweep the cache is an unbounded disk leak.

What never to prune

Never run docker image prune -a on a BenchPress host. Untagged but still
useful lab images are 5.5 GB to 19.7 GB each, and rebuilding one costs an hour
and most of the free disk.

These are safe:

Troubleshooting

Reference

Related

• Golden images — what the build bakes into each of these images.
• Prerequisites — sizing the disk before any of this.
• Create a lab — the screen that orders a build.