Skip to content

Conversation

@olix0r
Copy link
Member

@olix0r olix0r commented Aug 12, 2025

The CLI build infrastructure is spread across a few intertwined scripts:

bin/docker-pull-binaries prepares binaries from release by loading, running,
and extracting the contents of the cli-bin container image. cli/Dockerfile is
responsible for building all of the binaries. Then, the release and integration
workflows build the cli via the bin/docker-build-cli-bin script, etc...

This change moves all of the packaging concerns into cli/Dockerfile, removing
the bin/docker-pull-binaries and bin/docker-build-cli-bin scripts. We
introduce two small helper workflow utilities:

  • workflows/cli-build.yml is a callable workflow that uses modern Github actions
    infrastructure (e.g., docker/build-push-action)
  • actions/cli-setup downloads and 'installs' the linux-amd64 CLI from the
    artifact produced by the cli-build workflow.

BREAKING CHANGE: We no longer publish a cli-bin image with releases. The typical
release artifacts, however, remain unchanged.

olix0r added 2 commits August 12, 2025 21:49
We are using some non-standard way to authenticate to ghcr.

This change updates our release workflow to use docker/login-action and to
authenticate with the standard github action token.
@olix0r olix0r changed the base branch from main to ver/ci-docker-login August 12, 2025 23:45
@olix0r olix0r changed the title Ver/cli build chore(ci): update CLI build pipeline Aug 12, 2025
@olix0r olix0r force-pushed the ver/cli-build branch 2 times, most recently from fd8818e to d8e1623 Compare August 13, 2025 00:07
Base automatically changed from ver/ci-docker-login to main August 13, 2025 14:41
The CLI build infrastructure is spread across a few intertwined scripts:

`bin/docker-pull-binaries`` prepares binaries from release by loading, running,
and extracting the contents of the cli-bin container image. `cli/Dockerfile` is
responsible for building all of the binaries. Then, the release and integration
workflows build the cli via the `bin/docker-build-cli-bin` script, etc...

This change moves all of the packaging concerns into `cli/Dockerfile`, removing
the `bin/docker-pull-binaries` and `bin/docker-build-cli-bin` scripts. We
introduce two small helper workflow utilities:

* `workflows/cli-build.yml`` is a callable workflow that uses modern Github actions
  infrastructure (e.g., docker/build-push-action)
* `actions/cli-setup` downloads and 'installs' the linux-amd64 CLI from the
  artifact produced by the cli-build workflow.

BREAKING CHANGE: We no longer publish a cli-bin image with releases. The typical
release artifacts, however, remain unchanged.
@olix0r olix0r changed the title chore(ci): update CLI build pipeline chore(ci)!: update CLI build pipeline Aug 13, 2025
@olix0r olix0r marked this pull request as ready for review August 13, 2025 22:44
@olix0r olix0r requested a review from a team as a code owner August 13, 2025 22:44
@olix0r olix0r merged commit e514bc1 into main Aug 15, 2025
146 of 156 checks passed
@olix0r olix0r deleted the ver/cli-build branch August 15, 2025 00:11
@cratelyn
Copy link
Member

i believe this may have introduced a regression.

when driving an edge release this week, i pushed an edge-25.8.4 tag on main as of 46104e2.

https://github.com/linkerd/linkerd2/actions/runs/17138575187/job/48621791295

 #13 [go-gen  3/11] COPY charts charts
#13 DONE 0.0s
#14 [go-gen  4/11] COPY jaeger jaeger
#14 DONE 0.0s
#15 [go-gen  5/11] COPY multicluster multicluster
#15 DONE 0.0s
#16 [go-gen  6/11] COPY viz viz
#16 DONE 0.0s
#17 [go-gen  7/11] COPY controller/k8s controller/k8s
#17 DONE 0.0s
#18 [go-gen  8/11] COPY controller/api controller/api
#18 DONE 0.0s
#19 [go-gen  9/11] COPY controller/gen controller/gen
#19 DONE 0.0s
#20 [go-gen 10/11] COPY pkg pkg
#20 DONE 0.0s
#21 [go-gen 11/11] RUN mkdir -p /out
#21 DONE 0.1s
#22 [build-darwin 1/2] RUN go build -o "/out/linkerd2-cli-edge-25.8.4-darwin" -tags prod -mod=readonly -ldflags "-s -w -X github.com/linkerd/linkerd2/pkg/version.Version=edge-25.8.4" ./cli
#22 ...
#23 [build-darwin-arm64 1/2] RUN go build -o "/out/linkerd2-cli-edge-25.8.4-darwin-arm64" -tags prod -mod=readonly -ldflags "-s -w -X github.com/linkerd/linkerd2/pkg/version.Version=edge-25.8.4" ./cli
#23 ...
#24 [build-linux-arm64 1/2] RUN go build -o "/out/linkerd2-cli-edge-25.8.4-linux-arm64" -tags prod -mod=readonly -ldflags "-s -w -X github.com/linkerd/linkerd2/pkg/version.Version=edge-25.8.4" ./cli
#24 ...
#25 [build-windows 1/2] RUN go build -o "/out/linkerd2-cli-edge-25.8.4-windows.exe" -tags prod -mod=readonly -ldflags "-s -w -X github.com/linkerd/linkerd2/pkg/version.Version=edge-25.8.4" ./cli
#25 ...
#26 [build-linux-amd64 1/2] RUN go build -o "/out/linkerd2-cli-edge-25.8.4-linux-amd64" -tags prod -mod=readonly -ldflags "-s -w -X github.com/linkerd/linkerd2/pkg/version.Version=edge-25.8.4" ./cli

the job seemed to get stuck here, making no progress until the job eventually seemed to time out.

cratelyn added a commit that referenced this pull request Aug 21, 2025
cratelyn added a commit that referenced this pull request Aug 21, 2025
This reverts commit e514bc1.

Signed-off-by: katelyn martin <[email protected]>
@cratelyn
Copy link
Member

 #28 [build-darwin-arm64 2/2] RUN openssl dgst -sha256 "/out/linkerd2-cli-edge-25.8.4-darwin-arm64" | awk '{print $2}' > "/out/linkerd2-cli-edge-25.8.4-darwin-arm64.sha256"
#28 0.213 /bin/sh: openssl: not found
#28 DONE 0.2s

#29 [build-darwin 2/2] RUN openssl dgst -sha256 "/out/linkerd2-cli-edge-25.8.4-darwin" | awk '{print $2}' > "/out/linkerd2-cli-edge-25.8.4-darwin.sha256"
#29 0.300 /bin/sh: openssl: not found
#29 DONE 0.3s

#30 [build-windows 2/2] RUN openssl dgst -sha256 "/out/linkerd2-cli-edge-25.8.4-windows.exe" | awk '{print $2}' > "/out/linkerd2-cli-edge-25.8.4-windows.exe.sha256"
#30 0.260 /bin/sh: openssl: not found
#30 DONE 0.3s

#22 [build-linux-arm64 1/2] RUN go build -o "/out/linkerd2-cli-edge-25.8.4-linux-arm64" -tags prod -mod=readonly -ldflags "-s -w -X github.com/linkerd/linkerd2/pkg/version.Version=edge-25.8.4" ./cli
#22 DONE 510.0s

#31 [build-linux-arm64 2/2] RUN openssl dgst -sha256 "/out/linkerd2-cli-edge-25.8.4-linux-arm64" | awk '{print $2}' > "/out/linkerd2-cli-edge-25.8.4-linux-arm64.sha256"
#31 0.177 /bin/sh: openssl: not found
#31 DONE 0.2s

cratelyn added a commit that referenced this pull request Nov 7, 2025
this helper script wrapping docker is used by various scripts in the
development workflow.

in #14360, we removed the
`cli-bin` image from our published artifacts.

Signed-off-by: katelyn martin <[email protected]>
cratelyn added a commit that referenced this pull request Nov 7, 2025
this commit removes mention of the cli from our `bin/docker-build`
script.

in #14360, we removed the cli
image from our published artifacts. moreover, invoking `bin/linkerd`
will call `bin/build-cli-bin`.

so, this commit removes both calls, to the now deleted
`bin/docker-build-cli-bin` script, as well as the `bin/build-cli-bin`
script. this lets the `bin/docker-build` script focus solely on docker
images.

Signed-off-by: katelyn martin <[email protected]>
cratelyn added a commit that referenced this pull request Nov 7, 2025
this image is used within CI, and broadly as a "layer" to place the cli
binaries in a final image.

the cli image itself however, is not currently intended for use directly
via `docker build`, and will consequently yield an error response if
built directly.

this commit adds a comment providing some information about this.

see #14360 for more.

Signed-off-by: katelyn martin <[email protected]>
cratelyn added a commit that referenced this pull request Nov 7, 2025
this branch makes changes to scripts in the `bin/` directory, to
restore the end-to-end local development workflow described in
`BUILD.md` here:
https://github.com/linkerd/linkerd2/blob/main/BUILD.md#comprehensive

now that we no longer include a `bin-cli` in our published artifacts,
we can remove that image from the list of images that the `_docker.sh`
helper script is aware of. this means we now refrain from (a)
building the image when `bin/docker-build` is run, and (b) loading the
image when `bin/image-load` is run.

some related comments are updated in this branch, while we are here.

---

* refactor: remove `cli-bin` from `_docker.sh` images

this helper script wrapping docker is used by various scripts in the
development workflow.

in #14360, we removed the
`cli-bin` image from our published artifacts.

Signed-off-by: katelyn martin <[email protected]>

* nit(bin): update `build-cli-bin` comment

this commit removes mention of docker from this comment. there is no
longer an equivalent `docker-build-cli-bin` script, so this is the way
that we build the cli binary.

Signed-off-by: katelyn martin <[email protected]>

* fix(bin): remove cli from `bin/docker-build`

this commit removes mention of the cli from our `bin/docker-build`
script.

in #14360, we removed the cli
image from our published artifacts. moreover, invoking `bin/linkerd`
will call `bin/build-cli-bin`.

so, this commit removes both calls, to the now deleted
`bin/docker-build-cli-bin` script, as well as the `bin/build-cli-bin`
script. this lets the `bin/docker-build` script focus solely on docker
images.

Signed-off-by: katelyn martin <[email protected]>

* docs(cli): a comment about cli image `ENTRYPOINT`

this image is used within CI, and broadly as a "layer" to place the cli
binaries in a final image.

the cli image itself however, is not currently intended for use directly
via `docker build`, and will consequently yield an error response if
built directly.

this commit adds a comment providing some information about this.

see #14360 for more.

Signed-off-by: katelyn martin <[email protected]>

---------

Signed-off-by: katelyn martin <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants