Skip to content

Stage New CI/CD Improvements. #259

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 63 commits into from
Dec 19, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
63 commits
Select commit Hold shift + click to select a range
75b15a2
### ChangeLog:
reactive-firewall Dec 7, 2024
47cc6c2
### ChangeLog:
reactive-firewall Dec 7, 2024
7087a02
### ChangeLog:
reactive-firewall Nov 30, 2024
ac23904
### ChangeLog:
reactive-firewall Nov 30, 2024
a31a692
[DOCUMENTATION] Reword heading to "Multicast Python Module" (- WIP #2…
reactive-firewall Dec 7, 2024
1c6294f
[PATCH] minor tweak for performance (- WIP PR #248 -)
reactive-firewall Dec 8, 2024
d8bd1de
[CI] Fixed regression in code-style for the TOX job.
reactive-firewall Dec 8, 2024
84d4b65
[REFACTOR] Implement SOLID refactor of tests/check_pip (- WIP #190 -)
reactive-firewall Dec 9, 2024
04af69c
[TESTING] updated file permissions (- WIP #190 -)
reactive-firewall Dec 9, 2024
5eb52ff
[HOTFIX] (deps-dev): Bump markdownlint
dependabot[bot] Dec 10, 2024
28005c4
Merge pull request #250 from reactive-firewall/dependabot/npm_and_yar…
dependabot[bot] Dec 11, 2024
74e526a
[DOCUMENTATION] Implemented #240
reactive-firewall Dec 11, 2024
a146830
[FEATURE] Environment Variable Configuration Module (- WIP #31 -)
reactive-firewall Dec 12, 2024
7bab82b
[CI] fix for regression in CI EXTRAS-FOR-PIP
reactive-firewall Dec 12, 2024
0343dd7
[PATCH] Minor tweak as discussed in review #256
reactive-firewall Dec 12, 2024
8e0181b
[PATCH] Debug Part 2 of many (- WIP PR #253 -)
reactive-firewall Dec 12, 2024
d108c17
[PATCH] Debug part 3 of many (- WIP PR #253 -)
reactive-firewall Dec 12, 2024
c29c0a8
[DOCUMENTATION] Improvements as discussed in review (- WIP PR #256 -)
reactive-firewall Dec 12, 2024
2074c0d
[DOCUMENTATION] Cleaned up documentation style a bit.
reactive-firewall Dec 12, 2024
bea7415
[DOCUMENTATION] improved CI statistics with new metrics.
reactive-firewall Dec 12, 2024
77b44a6
[CI] Refactored CI to use Environments (- WIP #177 -)
reactive-firewall Dec 13, 2024
294bc0d
[REGRESSION] Debug part 1 of many for #117. :see_no_evil:
reactive-firewall Dec 13, 2024
0170554
[REGRESSION] Debug Part 2 of many for #177
reactive-firewall Dec 13, 2024
3c76d38
[REGRESSIONS] Debug part 3 for #177
reactive-firewall Dec 13, 2024
93c59c8
[REGRESSION] Debug part 4 of many for #177
reactive-firewall Dec 13, 2024
e47d528
[REGRESSION] Debug part 5 of many for #177
reactive-firewall Dec 13, 2024
e33dd4b
[REGRESSION] Debug part 6 of many for #177 :see_no_evil:
reactive-firewall Dec 13, 2024
120f9b4
[REGRESSION] Debug part 7 of many for #177 :see_no_evil:
reactive-firewall Dec 13, 2024
a755fea
[REGRESSION] Debug part 8 for #177 :see_no_evil:
reactive-firewall Dec 13, 2024
658a8e8
[CONFIG] Debug part 9 of many for #177 :see_no_evil:
reactive-firewall Dec 13, 2024
b4a59fc
[PATCH] Implement single file SOLID improvements for #190
reactive-firewall Dec 13, 2024
aa6b012
[PATCH] Pivioted and returned to single file design (- WIP #190 -)
reactive-firewall Dec 13, 2024
6e34a40
[CI] Fix for minor typo from refactoring (- WIP #190 -)
reactive-firewall Dec 13, 2024
5cd4bc8
[REGRESSION] Possible fix for `check_pip` (- WIP #190 & PR #253 -)
reactive-firewall Dec 13, 2024
73a5a6d
[PATCH] Minor tweak to `check_pip` (- WIP #190 -)
reactive-firewall Dec 14, 2024
d04fbc3
[CI] Ignore `chardet` if installed in CI already (- WIP #190 -)
reactive-firewall Dec 14, 2024
2a19694
[CI] Mitigate CI/CD site-customized packages from `check_pip` testing…
reactive-firewall Dec 14, 2024
f42e647
[REGRESSION] Fix for typo regression (- WIP #190 -)
reactive-firewall Dec 14, 2024
eedf835
[CI] Revert last change-set to fix regression (- WIP #190 -)
reactive-firewall Dec 14, 2024
0e7288b
[DOCUMENTATION] Added a few more badges :hear_no_evil: (- WIP PR #257 -)
reactive-firewall Dec 14, 2024
fdaffda
[CI] Debug part A of many for #177
reactive-firewall Dec 14, 2024
17974f0
[DOCUMENTATION] Minor tweak for markdown style and wording for CI.md …
reactive-firewall Dec 14, 2024
ad75b9b
[SECURITY] remove embedding token from codecov graphic.
reactive-firewall Dec 14, 2024
9e1000a
[MERGE] Version Bump to v2.0.3
reactive-firewall Dec 14, 2024
9d48e66
[CI] Debugging new refactored CI (- WIP #177 -)
reactive-firewall Dec 14, 2024
6b3320f
[CI] Tweaking the new CI overhaul a bit (- WIP #177 -)
reactive-firewall Dec 14, 2024
76fc182
[CI] Minor tweak for extra tests #177
reactive-firewall Dec 14, 2024
8041341
[CI] Fix for regression #177
reactive-firewall Dec 14, 2024
094249b
[HOTFIX] Release Candidate from `v2.0.3-rc-1` to `v2.0.3-rc-2`
reactive-firewall Dec 15, 2024
207f34f
[HOTFIX] Release Candidate version bump to `v2.0.3-rc-3`
reactive-firewall Dec 15, 2024
e098346
[HOTFIX] Workaround for Deep-source complaining about PYL-W1508
reactive-firewall Dec 15, 2024
0e446d7
[DOUMENTATION] Minor clearification regarding logo.svg and Multicast …
reactive-firewall Dec 15, 2024
046188a
[HOTFIX] Suppress TCV-002 from defensive code.
reactive-firewall Dec 15, 2024
c6bb5f6
[HOTFIX] Suppress more TCV-002 noise from deepsource
reactive-firewall Dec 15, 2024
46cbfa8
[StepSecurity] ci: Harden GitHub Actions
step-security-bot Dec 16, 2024
9ceacca
[MERGE] step-security-bot-stepsecurity_remediation_1734311585
reactive-firewall Dec 16, 2024
c54dab2
[STYLE] Minor changes found in review (- WIP PR #263 -)
reactive-firewall Dec 16, 2024
c6aa52b
[DOCUMENTATION] added docstring for last change.
reactive-firewall Dec 16, 2024
f27f4c3
[MERGE] Release Candidate `v2.0.3-rc-3` Staging.
reactive-firewall Dec 16, 2024
3f247e6
[UPDATE] (deps): Bump github/codeql-action in /.github/workflows
dependabot[bot] Dec 17, 2024
ae484f6
[UPDATE] (deps): Bump ossf/scorecard-action in /.github/workflows
dependabot[bot] Dec 17, 2024
a109e7a
Merge pull request #269 from reactive-firewall/dependabot/github_acti…
dependabot[bot] Dec 17, 2024
fe5643c
Merge pull request #267 from reactive-firewall/dependabot/github_acti…
dependabot[bot] Dec 17, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
169 changes: 169 additions & 0 deletions .github/workflows/CI-BUILD.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,169 @@
---
name: CI-BUILD
# Continuous Integration workflow for building, the project
#
# Jobs included:
# - BUILD: Ensures the project compiles correctly
# - BOOTSTRAP: Tests installation across Python versions and locales
#
# Required Secrets:
# NONE

on: # yamllint disable-line rule:truthy
push:
branches: ["**"] # matches any branch
tags: ["v*"]

# Declare default permissions as none.
permissions: {}

env:
ENVIRONMENT: ${{ (github.ref == 'refs/heads/stable' || startsWith(github.ref, 'refs/tags/v')) && 'Deployment' || (github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/heads/feature') || startsWith(github.ref, 'refs/heads/patch-') || startsWith(github.ref, 'refs/tags/v')) && 'Testing' || 'Experimenting' }}

jobs:
BUILD:
permissions:
actions: read
contents: read
statuses: write
packages: none
pull-requests: read
security-events: none
if: ${{ !cancelled() && (github.repository == 'reactive-firewall/multicast') }}
runs-on: ubuntu-latest
environment: ${{ (github.ref == 'refs/heads/stable' || startsWith(github.ref, 'refs/tags/v')) && 'Deployment' || (github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/heads/feature') || startsWith(github.ref, 'refs/heads/patch-') || startsWith(github.ref, 'refs/tags/v')) && 'Testing' || 'Experimenting' }}
defaults:
run:
shell: bash
env:
LANG: "en_US.UTF-8"
outputs:
build_status: ${{ steps.build.outcome }}
steps:
- name: Checkout repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- uses: actions/setup-python@0b93645e9fea7318ecaed2b359559ac225c90a2b # v5.3.0
with:
python-version: "3.12"
- name: Pre-Clean
id: clean
run: make -j1 -f Makefile purge 2>/dev/null || true
- name: Test Build
id: build
run: make -j1 -f Makefile build
- name: Post-Clean
id: post
run: make -j1 -f Makefile purge || true

BOOTSTRAP:
permissions:
actions: read
contents: read
statuses: write
packages: none
pull-requests: read
security-events: none
if: ${{ !cancelled() }}
needs: BUILD
runs-on: ubuntu-latest
environment: ${{ needs.BUILD.environment }}
defaults:
run:
shell: bash
timeout-minutes: 5
continue-on-error: ${{ matrix.experimental }}
strategy:
fail-fast: false
matrix:
python-version: ["3.10", "3.11", "3.12"]
lang-var: ["de.utf-8", "jp.utf-8"]
experimental: [true]
include:
- python-version: "3.10"
lang-var: "de.utf-8"
experimental: false
- python-version: "3.10"
lang-var: "jp.utf-8"
experimental: false
- python-version: "3.10"
lang-var: "en_US.utf-8"
experimental: false
- python-version: "3.11"
lang-var: "en_US.utf-8"
experimental: false
- python-version: "3.11"
lang-var: "en_US.utf-8"
experimental: false
- python-version: "3.12"
lang-var: "en_US.utf-8"
experimental: false
outputs:
bootstrap_status: ${{ steps.bootstrap.outcome }}
env:
PYTHON_VERSION: ${{ matrix.python-version }}
LANG: ${{ matrix.lang-var }}
steps:
- name: Checkout repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Set up Python
uses: actions/setup-python@0b93645e9fea7318ecaed2b359559ac225c90a2b # v5.3.0
with:
python-version: ${{ matrix.python-version }}
- name: Set up dependencies
run: |
pip install --upgrade "pip>=24.3.1" "setuptools>=75.0" "wheel>=0.44" "build>=1.2.1";
pip install -r ./requirements.txt ;
- name: Pre-build
id: bootstrap
run: |
make -j1 -f Makefile clean || true ;
make -j1 -f Makefile build ;
shell: bash
- name: Summerize Building
id: sumerize-py-build
run: |
echo "- Building works on python version ${{ matrix.python-version }}" >> $GITHUB_STEP_SUMMARY
if: ${{ success() }}
shell: bash
- name: Run Tests
id: test-user-install
run: make -j1 -f Makefile user-install ;
shell: bash
- name: Summerize Install
id: sumerize-user-install
run: |
echo "- User Installing works on python version ${{ matrix.python-version }}" >> $GITHUB_STEP_SUMMARY
if: ${{ success() }}
shell: bash
- name: Test Info
id: test-info
run: python -m setup --name --version --license || true ;
- name: Post-Clean
id: post-bootstrap
run: |
make -j1 -f Makefile purge || true ;
make -j1 -f Makefile clean || true ;
if: ${{ always() }}
shell: bash

BUILD_STATUS:
permissions:
actions: read
pull-requests: read
needs: [BUILD, BOOTSTRAP]
runs-on: ubuntu-latest
if: ${{ !cancelled() }}
outputs:
didBUILD: ${{ steps.check_status.outputs.build_success }}
steps:
- id: check_status
run: |
if [[ "${{ needs.BUILD.result }}" == "success" && "${{ needs.BOOTSTRAP.result }}" == "success" ]]; then
echo "build_success=true" >> $GITHUB_OUTPUT
else
echo "build_success=false" >> $GITHUB_OUTPUT
fi
86 changes: 86 additions & 0 deletions .github/workflows/CI-DOCS.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
---
name: CI-DOCS
# Continuous Integration workflow for Testing Documentation Building.
#
# Jobs included:
# - DOCS: Builds and lints documentation
#
# Required Secrets:
# - DOCS_BUILD_REF: Automatic

permissions: {}

on: # yamllint disable-line rule:truthy
push:
branches: ["main", "master", "stable"]
tags: ["v*"]
pull_request:
types: [opened, reopened, ready_for_review]
branches: ["feature-*", "patch-*", "HOTFIX-*"]
pull_request_target:
types: [opened, edited, reopened, ready_for_review]
branches: ["main", "master", "stable"]

jobs:
DOCS:
permissions:
actions: read
contents: read
statuses: write
packages: none
pull-requests: read
security-events: none
environment: ${{ (github.ref == 'refs/heads/stable' || startsWith(github.ref, 'refs/tags/v')) && 'Deployment' || (github.ref == 'refs/heads/master') && 'Testing' || 'Experimenting' }}
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
python-version: ["3.10", "3.11", "3.12"]
env:
OS: ${{ matrix.os }}
PYTHON_VERSION: ${{ matrix.python-version }}
LANG: "en_US.utf-8"
LC_CTYPE: "en_US.utf-8"
DOCS_BUILD_REF: ${{ github.sha }}
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@0b93645e9fea7318ecaed2b359559ac225c90a2b # v5.3.0
with:
python-version: ${{ matrix.python-version }}
- name: Fix braindead windows ${{ matrix.python-version }} on ${{ matrix.os }}
if: ${{ !cancelled() && runner.os == 'Windows' }}
run: python -m pip install --upgrade pip
- name: Install dependencies for python ${{ matrix.python-version }} on ${{ matrix.os }}
run: |
pip install --upgrade "pip>=24.3.1" "setuptools>=75.0" "wheel>=0.44" "build>=1.2.1";
pip install -r ./requirements.txt ;
pip install -r ./tests/requirements.txt || true ;
- name: Pre-Clean
id: clean-prep
run: make -j1 -f Makefile clean ;
- name: Pre-build for Python ${{ matrix.python-version }} on ${{ matrix.os }}
run: make -j1 -f Makefile build ;
if: ${{ success() }}
- name: Generate documentation with py${{ matrix.python-version }} on ${{ matrix.os }}
run: make -j1 -f Makefile build-docs 2>&1 >> $GITHUB_STEP_SUMMARY ;
if: ${{ !cancelled() }}
- name: Lint documentation
run: |
pip install --upgrade sphinx-lint
sphinx-lint docs/ 2>&1 >> $GITHUB_STEP_SUMMARY || true ;
if: ${{ !cancelled() }}
- name: Upload Docs Artifact with Python ${{ matrix.python-version }} on ${{ matrix.os }}
uses: actions/upload-artifact@b4b15b8c7c6ac21ea08fcf65892d2ee8f75cf882 # v4.4.3
with:
name: Multicast-Documentation-${{ github.sha }}-${{ matrix.os }}-${{ matrix.python-version }}
path: ./docs/www/
if-no-files-found: ignore
- name: Post-purge
id: post-uninstall
run: make -j1 -f Makefile purge || true ;
if: ${{ !cancelled() }}
- name: Post-Clean
id: post-end
run: make -j1 -f Makefile clean || true ;
if: ${{ !cancelled() }}
117 changes: 117 additions & 0 deletions .github/workflows/CI-MATs.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
---
name: CI-MATs
# Continuous Integration workflow for Minimal Acceptance Tests.
#
# Jobs included:
# - MATS: Runs Machine Acceptance Tests
#
# Required Secrets:
# NONE

on: # yamllint disable-line rule:truthy
workflow_run:
workflows: ["CI-BUILD"]
types:
- completed

# Declare default permissions as none.
permissions: {}

jobs:
check_build:
permissions:
actions: read
pull-requests: read
checks: write
runs-on: ubuntu-latest
env:
GH_TOKEN: ${{ github.token }}
outputs:
should_run: ${{ steps.check.outputs.should_run }}
build_environment: ${{ steps.get_env.outputs.environment }}
steps:
- id: check
run: |
if [[ "${{ github.event.workflow_run.conclusion }}" == "success" ]]; then
echo "should_run=true" >> $GITHUB_OUTPUT
else
echo "should_run=false" >> $GITHUB_OUTPUT
fi
- id: get_env
run: |
ENV_VALUE=$(gh api "${{ github.event.workflow_run.artifacts_url }}" --jq '.environment')
if [[ -n "$ENV_VALUE" ]]; then
echo "environment=$ENV_VALUE" >> $GITHUB_OUTPUT
else
echo "environment=Experimenting" >> $GITHUB_OUTPUT # Default fallback
fi

MATS:
permissions:
actions: read
contents: read
statuses: write
needs: check_build
if: ${{ !cancelled() && (needs.check_build.outputs.should_run == 'true') }}
runs-on: ubuntu-latest
environment: ${{ needs.check_build.outputs.build_environment }}
defaults:
run:
shell: bash
timeout-minutes: 8
strategy:
matrix:
python-version: ["3.10", "3.11", "3.12"]
outputs:
mats_status: ${{ steps.tests.outcome }}
env:
PYTHON_VERSION: ${{ matrix.python-version }}
LANG: "en_US.utf-8"
steps:
- name: Checkout repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@0b93645e9fea7318ecaed2b359559ac225c90a2b # v5.3.0
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies for ${{ matrix.python-version }}
run: |
pip install --upgrade "pip>=24.3.1" "setuptools>=75.0" "wheel>=0.44" "build>=1.2.1";
pip install -r ./requirements.txt ;
pip install -r ./tests/requirements.txt || true ;
pip install --upgrade -r ./docs/requirements.txt || true ;
- name: Pre-Clean
id: clean
run: make -j1 -f Makefile clean || true ;
- name: Run Tests for python ${{ matrix.python-version }}
id: tests
run: make -j1 -f Makefile test ;
- name: Summerize MATs for python ${{ matrix.python-version }}
id: sumerize-mats
run: |
echo "- MATS works on python version ${{ matrix.python-version }}" >> $GITHUB_STEP_SUMMARY
if: ${{ success() }}
- name: Post-Clean
id: post
run: make -j1 -f Makefile clean || true ;
if: ${{ always() }}

MATS_STATUS:
permissions:
actions: read
needs: [check_build, MATS]
runs-on: ubuntu-latest
if: ${{ !cancelled() }}
outputs:
passedMATs: ${{ steps.check_status.outputs.mats_success }}
didBUILD: ${{ needs.check_build.outputs.should_run }}
steps:
- id: check_status
run: |
if [[ "${{ needs.MATS.result }}" == "success" ]]; then
echo "mats_success=true" >> $GITHUB_OUTPUT
else
echo "mats_success=false" >> $GITHUB_OUTPUT
fi
2 changes: 1 addition & 1 deletion .github/workflows/Labeler.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,6 @@ jobs:
security-events: none
runs-on: ubuntu-latest
steps:
- uses: actions/labeler@v5
- uses: actions/labeler@8558fd74291d67161a8a78ce36a881fa63b766a9 # v5.0.0
with:
repo-token: "${{ secrets.GITHUB_TOKEN }}"
Loading
Loading