Skip to content

Commit 756b394

Browse files
mowiesdouglascamata
mowies
and
douglascamata
authored
Add nightly releases (#1016)
* add nightly release setup with pipeline and goreleaser settings update goreleaser config with nightly settings use different token to push nightly tag revert core ci, change to pushing nightly tags prepare otelcol goreleaser setup for testing add new pipeline * revert otelcol dockers * fix pipeline try to move nightly check to distro release pipelines enable otelcol windows re-enable container manifests try disabling image manifests fix env var remove quotes switch to env var for ephermal tag try to exchange latest image tag when building nightly enable container images with personal user only keep a single pre-release add nightly setup to contrib build-only config remove otelcol and otelcol-contrib dockers see if otelcol-contrib works restore otelcol dockers * cherry pick fixes? * more setup things * revert some debugging changes * revert goreleaser debugging changes * revert debugging changes for builder and opamp * revert debugging changes in dockerfiles * remove force pushes * rename GORELEASER_PREVIOUS_TAG * filter tags before taking the right one for all release pipelines * adjust doc comments * add missing newlines * update goreleaser to correct version Co-authored-by: Douglas Camata <[email protected]> * revert change of github token in base-release * add comments * use otelbot short time token for pushing nightly tags * set specific content write permission for new token * disable publishing nightly releases * revert changes of github tokens * revert changes of github tokens * set up otelbot app committer name and email correctly * create issue on failed nightly release * add note about other distro releases * add changelog entry --------- Co-authored-by: Douglas Camata <[email protected]>
1 parent d53d6b4 commit 756b394

File tree

20 files changed

+296
-31
lines changed

20 files changed

+296
-31
lines changed

‎.chloggen/mowies-nightly-releases.yaml‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
# Use this changelog template to create an entry for release notes.
2+
3+
# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
4+
change_type: enhancement
5+
6+
# The name of the component, or a single word describing the area of concern, (e.g. otlpreceiver)
7+
component: releases
8+
9+
# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
10+
note: Introduce nightly releases
11+
12+
# One or more tracking issues or pull requests related to the change
13+
issues: [1016]
14+
15+
# (Optional) One or more lines of additional information to render under the primary note.
16+
# These lines will be padded with 2 spaces and then inserted directly into the document.
17+
# Use pipe (|) for multiline entries.
18+
subtext:
19+
20+
# Optional: The change log or logs in which this entry should be included.
21+
# e.g. '[user]' or '[user, api]'
22+
# Include 'user' if the change is relevant to end users.
23+
# Include 'api' if there is a change to a library API.
24+
# Default: '[user]'
25+
change_logs: [user]
26+

‎.github/workflows/base-release.yaml‎

Lines changed: 62 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -16,13 +16,19 @@ on:
1616
required: false
1717
type: string
1818
default: ubuntu-24.04
19+
nightly:
20+
required: false
21+
type: string
22+
default: false
1923

2024
env:
2125
# renovate: datasource=github-releases depName=goreleaser/goreleaser-pro
2226
GORELEASER_PRO_VERSION: v2.11.0
2327

2428
jobs:
2529
prepare:
30+
outputs:
31+
PREVIOUS_RELEASE_TAG: ${{ steps.prev-tag.outputs.PREVIOUS_RELEASE_TAG }}
2632
strategy:
2733
matrix:
2834
GOOS: ${{ fromJSON(inputs.goos) }}
@@ -72,8 +78,25 @@ jobs:
7278
go-version: "~1.24"
7379
check-latest: true
7480

81+
- name: Set goreleaser last tag reference in case of non-nightly release
82+
id: prev-tag
83+
if: inputs.nightly != 'true'
84+
# find previous tag by filtering out nightly tags and binary release (OCB, OpAMP) tags and then choosing the
85+
# second to last tag (last one is the current release)
86+
run: |
87+
prev_tag=$(git tag | grep -v "cmd" | grep -v "nightly" | sort -r --version-sort | head -n 2 | tail -n 1)
88+
echo "PREVIOUS_RELEASE_TAG=$prev_tag" >> "$GITHUB_OUTPUT"
89+
90+
- name: Set nightly enabled
91+
id: nightly-check
92+
if: inputs.nightly == 'true'
93+
run: |
94+
echo "NIGHTLY_FLAG=--nightly" >> "$GITHUB_OUTPUT"
95+
7596
- name: Generate distribution sources
7697
run: make generate-sources
98+
env:
99+
DISTRIBUTIONS: ${{ inputs.distribution }}
77100

78101
- name: Log into Docker.io
79102
uses: docker/login-action@74a5d142397b4f367a81961eba4e8cd7edddf772 # v3.4.0
@@ -101,7 +124,7 @@ jobs:
101124
distribution: goreleaser-pro
102125
version: ${{ env.GORELEASER_PRO_VERSION }}
103126
workdir: distributions/${{ inputs.distribution }}
104-
args: release --clean --split --timeout 2h --config .goreleaser-build.yaml --release-header-tmpl=../../.github/release-template.md
127+
args: release --clean --split --timeout 2h --config .goreleaser-build.yaml --release-header-tmpl=../../.github/release-template.md ${{ steps.nightly-check.outputs.NIGHTLY_FLAG }}
105128
env:
106129
GOOS: ${{ matrix.GOOS }}
107130
GOARCH: ${{ matrix.GOARCH }}
@@ -111,6 +134,7 @@ jobs:
111134
GORELEASER_KEY: ${{ secrets.GORELEASER_KEY }}
112135
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
113136
COSIGN_YES: true
137+
GORELEASER_CURRENT_TAG: ${{ github.ref_name }}
114138

115139
- name: Move built artifacts
116140
if: inputs.distribution == 'otelcol-contrib'
@@ -127,7 +151,7 @@ jobs:
127151
distribution: goreleaser-pro
128152
version: ${{ env.GORELEASER_PRO_VERSION }}
129153
workdir: distributions/${{ inputs.distribution }}
130-
args: release --clean --split --timeout 2h --release-header-tmpl=../../.github/release-template.md
154+
args: release --clean --split --timeout 2h --release-header-tmpl=../../.github/release-template.md ${{ steps.nightly-check.outputs.NIGHTLY_FLAG }}
131155
env:
132156
GOOS: ${{ matrix.GOOS }}
133157
GOARCH: ${{ matrix.GOARCH }}
@@ -143,7 +167,7 @@ jobs:
143167
if-no-files-found: error
144168

145169
release:
146-
name: ${{ inputs.distribution }} Release
170+
name: Release
147171
runs-on: ${{ inputs.runner_os }}
148172
needs: prepare
149173

@@ -232,3 +256,38 @@ jobs:
232256
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
233257
COSIGN_YES: true
234258
GORELEASER_KEY: ${{ secrets.GORELEASER_KEY }}
259+
GORELEASER_PREVIOUS_TAG: ${{ needs.prepare.outputs.PREVIOUS_RELEASE_TAG }}
260+
GORELEASER_CURRENT_TAG: ${{ github.ref_name }}
261+
262+
create-issue:
263+
name: Create GitHub issue on failed nightly release
264+
runs-on: ubuntu-latest
265+
needs: [release]
266+
if: failure() && inputs.nightly == 'true'
267+
permissions:
268+
issues: write
269+
env:
270+
issue-file: "nightly-release-failure.md"
271+
steps:
272+
- name: Formulate issue
273+
run: |
274+
# create a markdown file that contains details about the failure
275+
echo "---" > ${{ env.issue-file }}
276+
echo "title: Nightly Release Failed" >> ${{ env.issue-file }}
277+
echo "labels: release:blocker" >> ${{ env.issue-file }}
278+
echo "---" >> ${{ env.issue-file }}
279+
echo "The nightly release failed in the following GitHub actions run." >> ${{ env.issue-file }}
280+
echo "* Link to run: $GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID" >> ${{ env.issue-file }}
281+
echo "* Please also check the other distro release runs related to this nightly release" >> ${{ env.issue-file }}
282+
echo "* Triggered by: Nightly scheduled release" >> ${{ env.issue-file }}
283+
echo "" >> ${{ env.issue-file }}
284+
echo "Note: This issue was auto-generated from [base-release.yaml](https://github.com/open-telemetry/opentelemetry-collector-releases/blob/main/.github/workflows/base-release.yaml)" >> ${{ env.issue-file }}
285+
286+
- name: Create or update issue
287+
uses: JasonEtco/create-an-issue@1b14a70e4d8dc185e5cc76d3bec9eab20257b2c5 # v2.9.2
288+
env:
289+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
290+
with:
291+
update_existing: true
292+
search_existing: open
293+
filename: ${{ env.issue-file }}

‎.github/workflows/ci-goreleaser-core.yaml‎

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
name: CI - Core - GoReleaser
22

33
on:
4-
merge_group:
4+
merge_group:
55
push:
66
branches: [main]
77
paths:
@@ -34,12 +34,12 @@ jobs:
3434
secrets: inherit
3535

3636
package-tests:
37-
name: Package tests
38-
needs: check-goreleaser
39-
uses: ./.github/workflows/base-package-tests.yaml
40-
with:
41-
distribution: otelcol
42-
type: '[ "deb", "rpm" ]'
37+
name: Package tests
38+
needs: check-goreleaser
39+
uses: ./.github/workflows/base-package-tests.yaml
40+
with:
41+
distribution: otelcol
42+
type: '[ "deb", "rpm" ]'
4343

4444
msi-tests:
4545
name: MSI tests

‎.github/workflows/nightly-release.yaml‎

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
name: Nightly Release
2+
3+
on:
4+
workflow_dispatch:
5+
schedule:
6+
- cron: "0 2 * * *" # release nightly at 2AM
7+
8+
permissions:
9+
contents: read
10+
11+
env:
12+
# renovate: datasource=github-tags depName=goreleaser-pro packageName=goreleaser/goreleaser-pro
13+
GORELEASER_PRO_VERSION: v2.11.0
14+
15+
jobs:
16+
nightly-release:
17+
name: Nightly Release
18+
runs-on: ubuntu-latest
19+
20+
steps:
21+
- uses: actions/create-github-app-token@df432ceedc7162793a195dd1713ff69aefc7379e # v2.0.6
22+
id: otelbot-token
23+
with:
24+
app-id: ${{ vars.OTELBOT_COLLECTOR_RELEASES_APP_ID }}
25+
private-key: ${{ secrets.OTELBOT_COLLECTOR_RELEASES_PRIVATE_KEY }}
26+
permission-contents: write
27+
28+
# The next 2 steps are taken from # from https://github.com/actions/create-github-app-token#create-a-git-committer-string-for-an-app-installation
29+
- name: Get GitHub App User ID
30+
id: get-user-id
31+
run: echo "user-id=$(gh api "/users/${{ steps.otelbot-token.outputs.app-slug }}[bot]" --jq .id)" >> "$GITHUB_OUTPUT"
32+
env:
33+
GH_TOKEN: ${{ steps.otelbot-token.outputs.token }}
34+
35+
- name: Set up commit author name and email
36+
id: committer
37+
run: |
38+
echo "name=${{ steps.otelbot-token.outputs.app-slug }}[bot]" >> "$GITHUB_OUTPUT"
39+
echo "email=${{ steps.get-user-id.outputs.user-id }}+${{ steps.otelbot-token.outputs.app-slug }}[bot]@users.noreply.github.com>" >> "$GITHUB_OUTPUT"
40+
41+
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
42+
with:
43+
fetch-depth: 0
44+
token: ${{ steps.otelbot-token.outputs.token }}
45+
46+
- name: 'Push new tag'
47+
run: |
48+
git config user.name "${{ steps.committer.outputs.name }}"
49+
git config user.email "${{ steps.committer.outputs.email }}"
50+
51+
# A previous release was created using a lightweight tag
52+
# git describe by default includes only annotated tags
53+
# git describe --tags includes lightweight tags as well
54+
DESCRIBE=`git tag -l --sort=-v:refname | grep -v cmd | grep -v nightly | head -n 1` # list tags except the ones containing cmd or nightly and then take the latest one
55+
MAJOR_VERSION=`echo $DESCRIBE | awk '{split($0,a,"."); print a[1]}'` # take just the major version digits, e.g. "v0"
56+
MINOR_VERSION=`echo $DESCRIBE | awk '{split($0,a,"."); print a[2]}'` # take just the minor version digits, e.g. "130"
57+
MINOR_VERSION="$((${MINOR_VERSION} + 1))" # bump minor version
58+
TAG="${MAJOR_VERSION}.${MINOR_VERSION}.0-nightly.$(date +'%Y%m%d%H%M')"
59+
git tag -a $TAG -m "$TAG: nightly build"
60+
git push origin $TAG

‎.github/workflows/release-builder.yaml‎

Lines changed: 35 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -22,23 +22,51 @@ jobs:
2222
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
2323
with:
2424
fetch-depth: 0
25+
26+
- name: Set goreleaser last tag reference in case of non-nightly release
27+
id: prev-tag
28+
if: ${{ !contains(github.ref, '-nightly') }}
29+
# find previous tag by taking only OCB tags, filtering out nightly tags and then choosing the
30+
# second to last tag (last one is the current release)
31+
run: |
32+
prev_tag=$(git tag | grep "cmd/builder" | grep -v "nightly" | sort -r --version-sort | head -n 2 | tail -n 1)
33+
echo "PREVIOUS_RELEASE_TAG=$prev_tag" >> "$GITHUB_OUTPUT"
34+
35+
- name: Set nightly enabled
36+
id: nightly-check
37+
if: ${{ contains(github.ref, '-nightly') }}
38+
run: |
39+
echo "NIGHTLY_FLAG=--nightly" >> "$GITHUB_OUTPUT"
40+
41+
- name: Set COLLECTOR_REF
42+
id: collector-ref
43+
run: |
44+
if [ '${{ contains(github.ref, '-nightly') }}' == 'true' ]; then
45+
echo "COLLECTOR_REF=main" >> "$GITHUB_OUTPUT"
46+
else
47+
echo "COLLECTOR_TAG=${{ github.ref }}" >> "$GITHUB_OUTPUT"
48+
fi
49+
2550
- name: Push cmd/builder Tag
2651
run: |
2752
tag="cmd/builder/${{ github.ref_name }}"
28-
message="Releasing ocb binaries for ${{ github.ref_name }}"
53+
message="Releasing ocb binaries for ${{ steps.collector-ref.outputs.COLLECTOR_REF }}"
2954
git config user.name "${GITHUB_ACTOR}"
3055
git config user.email "${GITHUB_ACTOR}@users.noreply.github.com"
3156
git tag -a "${tag}" -m "${message}"
3257
git push origin "${tag}"
58+
3359
- name: Checkout Core Repo
3460
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
3561
with:
3662
fetch-depth: 0
3763
repository: "open-telemetry/opentelemetry-collector"
38-
ref: ${{ github.ref_name }}
64+
ref: ${{ steps.collector-ref.outputs.COLLECTOR_REF }}
3965
path: ".core"
66+
4067
- name: Copy Dockerfile to Core Repo directory
4168
run: cp cmd/builder/Dockerfile .core/cmd/builder/Dockerfile
69+
4270
- uses: sigstore/cosign-installer@d58896d6a1865668819e1d91763c7751a165e159 # v3.9.2
4371
- uses: anchore/sbom-action/download-syft@7b36ad622f042cab6f59a75c2ac24ccb256e9b45 # v0.20.4
4472
- uses: docker/setup-qemu-action@29109295f81e9208d7d86ff1c6c12d2833863392 # v3.6.0
@@ -49,6 +77,7 @@ jobs:
4977
uses: actions/setup-go@d35c59abb061a4a6fb18e82ac0862c26744d6ab5 # v5.5.0
5078
with:
5179
go-version: "~1.24"
80+
5281
- name: Log into Docker.io
5382
uses: docker/login-action@74a5d142397b4f367a81961eba4e8cd7edddf772 # v3.4.0
5483
with:
@@ -60,14 +89,17 @@ jobs:
6089
registry: ghcr.io
6190
username: ${{ github.repository_owner }}
6291
password: ${{ secrets.GITHUB_TOKEN }}
92+
6393
- name: Run GoReleaser
6494
uses: goreleaser/goreleaser-action@9c156ee8a17a598857849441385a2041ef570552 # v6.3.0
6595
with:
6696
distribution: goreleaser-pro
6797
version: ${{ env.GORELEASER_PRO_VERSION }}
68-
args: release --clean -f cmd/builder/.goreleaser.yml
98+
args: release --clean -f cmd/builder/.goreleaser.yml ${{ steps.nightly-check.outputs.NIGHTLY_FLAG }}
6999
env:
70100
GORELEASER_KEY: ${{ secrets.GORELEASER_KEY }}
71101
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
72102
COSIGN_YES: true
73103
SKIP_SIGNS: false
104+
GORELEASER_PREVIOUS_TAG: ${{ steps.prev-tag.outputs.PREVIOUS_RELEASE_TAG }}
105+
GORELEASER_CURRENT_TAG: ${{ github.ref_name }}

‎.github/workflows/release-contrib.yaml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,10 +12,12 @@ jobs:
1212
distribution: otelcol-contrib
1313
goos: '[ "linux", "darwin" ]'
1414
goarch: '[ "386", "amd64", "arm64", "ppc64le", "arm", "s390x" ]'
15+
nightly: ${{ contains(github.ref, '-nightly') }}
1516
secrets: inherit
1617
permissions: write-all
1718
release-windows:
1819
name: Release Contrib (Windows)
20+
if: ${{ !contains(github.ref, '-nightly') }}
1921
uses: ./.github/workflows/base-release.yaml
2022
with:
2123
distribution: otelcol-contrib

‎.github/workflows/release-core.yaml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,10 +12,12 @@ jobs:
1212
distribution: otelcol
1313
goos: '[ "linux", "darwin" ]'
1414
goarch: '[ "386", "amd64", "arm64", "ppc64le", "arm", "s390x" ]'
15+
nightly: ${{ contains(github.ref, '-nightly') }}
1516
secrets: inherit
1617
permissions: write-all
1718
release-windows:
1819
name: Release Core (Windows)
20+
if: ${{ !contains(github.ref, '-nightly') }}
1921
uses: ./.github/workflows/base-release.yaml
2022
with:
2123
distribution: otelcol

‎.github/workflows/release-k8s.yaml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,10 +12,12 @@ jobs:
1212
distribution: otelcol-k8s
1313
goos: '[ "linux" ]'
1414
goarch: '[ "amd64", "arm64", "ppc64le", "s390x" ]'
15+
nightly: ${{ contains(github.ref, '-nightly') }}
1516
secrets: inherit
1617
permissions: write-all
1718
release-windows:
1819
name: Release k8s (Windows)
20+
if: ${{ !contains(github.ref, '-nightly') }}
1921
uses: ./.github/workflows/base-release.yaml
2022
with:
2123
distribution: otelcol-k8s

0 commit comments

Comments
 (0)