ENT-13777, ENT-13784: Added container-based CFEngine package builder by larsewi · Pull Request #2146 · cfengine/buildscripts

larsewi · 2026-03-19T14:34:14Z

Added build-in-container, a Python/Docker-based build tool that builds CFEngine packages inside containers using the existing build scripts
Supports Debian/Ubuntu platforms (ubuntu-20, ubuntu-22, ubuntu-24, debian-11, debian-12) with incremental builds via named Docker volumes. Currently only ubuntu-20 is tested (see ticket ENT-13777).

See build-in-container.md for full documentation.

cf-bottom · 2026-03-20T13:38:12Z

Thanks for submitting a pull request! Maybe @craigcomstock can review this?

olehermanse

Looks good overall, some comments and suggestions here and there.

build-in-container.md

olehermanse · 2026-03-30T10:51:30Z

build-in-container.py

@@ -0,0 +1,362 @@
+#!/usr/bin/env python3


IIRC craig said we already have a script for this, which is used in GH Actions. Might be good to align this, maybe delete the old one and start using the new one if appropriate.

Yepp, he probably means enterprise/ci/docker-build-package.sh. Although this script also runs tests, which we are trying to separate.

We can replace it after adding SFTP cache support to build-in-container.py.

build-in-container.md

olehermanse · 2026-03-30T11:09:21Z

build-in-container-inner.sh

+# === Build steps ===
+run_step "01-autogen" "$BASEDIR/buildscripts/build-scripts/autogen"
+run_step "02-install-dependencies" "$BASEDIR/buildscripts/build-scripts/install-dependencies"
+if [ "$EXPLICIT_ROLE" = "hub" ]; then
+    run_step "03-mission-portal-deps" install_mission_portal_deps
+fi
+run_step "04-configure" "$BASEDIR/buildscripts/build-scripts/configure"
+run_step "05-compile" "$BASEDIR/buildscripts/build-scripts/compile"
+run_step "06-package" "$BASEDIR/buildscripts/build-scripts/package"


Looks like we should rename build-scripts folder to steps / build-steps 😅.

craigcomstock · 2026-03-31T15:28:43Z

container/Dockerfile.debian

+
+ENV DEBIAN_FRONTEND=noninteractive
+
+# Build tools extracted from ci/cfengine-build-host-setup.cf (debian|ubuntu section)


yeah, it would be sad to copy/paste that content here so that now we have to maintain two locations again.

Yeah, I will look into these things

This is a bigger task. The policy does a lot of things we don't want in a container. I also see that we do the same in enterprise/ci/setup.sh. So we basically have three places where we maintain these dependencies. I think the ci/cfengine-build-host-setup.cf would need some refactoring.

Maybe we should do this in a separate ticket?

craigcomstock · 2026-03-31T15:29:31Z

container/Dockerfile.debian

+
+# Hub build tools: Node.js 20 LTS (system nodejs is too old for modern npm
+# packages that use the node: protocol), PHP, and Composer
+RUN curl -fsSL https://deb.nodesource.com/setup_20.x | bash - \


would be nice to stick with individual scripts we can maintain with new versions like https://github.com/cfengine/buildscripts/blob/master/ci/linux-install-jdk21.sh

build-in-container.py

Introduced build-in-container, a Python/Docker-based build system that builds CFEngine packages inside containers using the existing build scripts. Ticket: ENT-13777 Signed-off-by: Lars Erik Wik <lars.erik.wik@northern.tech>

Replaced the ListPlatformsAction custom argparse action with a simple --list-platforms store_true flag, and moved argument validation from argparse's required=True to manual post-parse checks. This allows future flags (e.g. --push-image) to bypass build-specific arguments like --project, --role, and --build-type. Signed-off-by: Lars Erik Wik <lars.erik.wik@northern.tech>

Signed-off-by: Lars Erik Wik <lars.erik.wik@northern.tech>

Each platform entry now includes a versioned image_tag field composed from the platform name and IMAGE_VERSION. The build_image() function uses this tag instead of constructing it locally. This prepares for registry-based image management where the tag includes a version for cache busting. Ticket: ENT-13784 Signed-off-by: Lars Erik Wik <lars.erik.wik@northern.tech>

The --push-image flag is a standalone image management operation that builds and pushes a container image to the registry. It only requires --platform (not --project/--role/--build-type). Ticket: ENT-13784 Signed-off-by: Lars Erik Wik <lars.erik.wik@northern.tech>

- registry_image_ref() returns the fully-qualified ghcr.io reference - pull_image() pulls from the registry, returns None on failure - push_image() tags a local image and pushes it to the registry Ticket: ENT-13784 Signed-off-by: Lars Erik Wik <lars.erik.wik@northern.tech>

- --push-image: builds with --no-cache and pushes to registry, then exits - --rebuild-image: builds locally, skips registry - Default: pulls from registry, falls back to local build on failure Ticket: ENT-13784 Signed-off-by: Lars Erik Wik <lars.erik.wik@northern.tech>

Added --push-image to argument tables, documented the registry pull/push workflow, fallback behavior, and toolchain update process. Ticket: ENT-13784 Signed-off-by: Lars Erik Wik <lars.erik.wik@northern.tech>

Manually triggered workflow that builds and pushes container images to ghcr.io. Builds all platforms in parallel via a matrix strategy. Ticket: ENT-13784 Signed-off-by: Lars Erik Wik <lars.erik.wik@northern.tech>

larsewi · 2026-04-01T16:09:07Z

Added container registry support since last review and tested it on my fork. Seems to work. Maybe we need to edit some permissions upstream. @olehermanse, @craigcomstock this is a rough WIP please let me know what you think.

craigcomstock · 2026-04-03T14:03:01Z

build-in-container.py


 log = logging.getLogger("build-in-container")

+IMAGE_REGISTRY = "ghcr.io/cfengine"


We do have a docker hub account FWIW. Not sure if we want to use either really...

What would you suggest instead?

craigcomstock · 2026-04-03T14:03:33Z

build-in-container.py

+    parser.add_argument(
+        "--push-image",
+        action="store_true",
+        help="Build image and push to registry, then exit",


are credentials required for pushing?

You can use the GITHUB_TOKEN. See https://github.com/docker/login-action?tab=readme-ov-file#github-container-registry

craigcomstock · 2026-04-03T14:04:56Z

build-in-container.md

+
+### Container registry
+
+Images are hosted at `ghcr.io/cfengine` and versioned via `IMAGE_VERSION` in


Here you might mention what credentials you need to push or to pull?

GITHUB_TOKEN to push (done through manually triggered workflow). To pull, no credentials needed as long as the images are made public (which we intend). You can push manually, but then you would need a PAT (classic)

Signed-off-by: Lars Erik Wik <lars.erik.wik@northern.tech>

larsewi force-pushed the container branch from 9fbc6ae to c66c0a0 Compare March 19, 2026 15:16

larsewi force-pushed the container branch from c66c0a0 to 70e99db Compare March 25, 2026 15:04

larsewi marked this pull request as ready for review March 25, 2026 15:31

larsewi requested review from craigcomstock and olehermanse March 25, 2026 15:31

olehermanse approved these changes Mar 30, 2026

View reviewed changes

craigcomstock approved these changes Mar 31, 2026

View reviewed changes

larsewi force-pushed the container branch from 70e99db to 7bdabae Compare March 31, 2026 16:06

larsewi marked this pull request as draft April 1, 2026 14:44

larsewi force-pushed the container branch 3 times, most recently from a950f51 to 945be75 Compare April 1, 2026 15:16

larsewi added 3 commits April 1, 2026 17:31

Added container-based CFEngine package builder

c995ac4

Introduced build-in-container, a Python/Docker-based build system that builds CFEngine packages inside containers using the existing build scripts. Ticket: ENT-13777 Signed-off-by: Lars Erik Wik <lars.erik.wik@northern.tech>

Extracted argument parsing into parse_args() function

57bf999

Signed-off-by: Lars Erik Wik <lars.erik.wik@northern.tech>

larsewi force-pushed the container branch 3 times, most recently from ff051d4 to 60bb3a1 Compare April 1, 2026 16:01

larsewi added 6 commits April 1, 2026 18:03

Updated documentation for container registry support

1859e0c

Added --push-image to argument tables, documented the registry pull/push workflow, fallback behavior, and toolchain update process. Ticket: ENT-13784 Signed-off-by: Lars Erik Wik <lars.erik.wik@northern.tech>

Added GitHub Actions workflow for building base images

3a5b5f4

Manually triggered workflow that builds and pushes container images to ghcr.io. Builds all platforms in parallel via a matrix strategy. Ticket: ENT-13784 Signed-off-by: Lars Erik Wik <lars.erik.wik@northern.tech>

larsewi force-pushed the container branch from 60bb3a1 to 3a5b5f4 Compare April 1, 2026 16:04

larsewi changed the title ~~ENT-13777: Added container-based CFEngine package builder~~ ENT-13777, ENT-13784: Added container-based CFEngine package builder Apr 1, 2026

craigcomstock reviewed Apr 3, 2026

View reviewed changes

Documented credentials and setup for container registry push/pull

fb44f60

Signed-off-by: Lars Erik Wik <lars.erik.wik@northern.tech>

larsewi marked this pull request as ready for review April 7, 2026 16:24


		ENV DEBIAN_FRONTEND=noninteractive

		# Build tools extracted from ci/cfengine-build-host-setup.cf (debian\|ubuntu section)


		log = logging.getLogger("build-in-container")

		IMAGE_REGISTRY = "ghcr.io/cfengine"


		### Container registry

		Images are hosted at `ghcr.io/cfengine` and versioned via `IMAGE_VERSION` in

Conversation

larsewi commented Mar 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cf-bottom commented Mar 20, 2026

Uh oh!

olehermanse left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

larsewi commented Apr 1, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

4 participants

larsewi commented Mar 19, 2026 •

edited

Loading