Compare commits

...
30 Commits
Author SHA1 Message Date
CI Bot 4f7b67203a chore: release v2.0.4 [skip ci] 2026-07-10 12:47:15 +00:00
brian a27e9e5763 Merge pull request '[fix] Include optional dependencies in security audit' (#50) from cursor/security-audit-all-extras into main
Test Python Package / unit-tests (push) Successful in 10s
Release on merge to main / release (push) Successful in 14s
Code Quality Pipeline / code-quality (push) Successful in 21s
Test Python Package / integration-tests (push) Successful in 20s
Test Python Package / coverage-report (push) Successful in 8s
Reviewed-on: LilleVemmelund/python-repositories#50
2026-07-10 14:47:01 +02:00
Brian Bjarke JensenandCursor fd0045d3e0 Include optional dependencies in security audit.
PR Title Check / check-title (pull_request) Successful in 5s
Test Python Package / unit-tests (pull_request) Successful in 12s
Code Quality Pipeline / code-quality (pull_request) Successful in 21s
Test Python Package / integration-tests (pull_request) Successful in 19s
Test Python Package / coverage-report (pull_request) Successful in 8s
Sync all extras before safety check so redis and minio vulnerabilities are scanned.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-10 14:44:35 +02:00
brian 515f4bbeae Merge pull request 'revert 707f11f578f007c7152582ea524b66dacea50111' (#49) from brian-patch-1 into main
Release on merge to main / release (push) Successful in 5s
Test Python Package / unit-tests (push) Successful in 10s
Code Quality Pipeline / code-quality (push) Successful in 20s
Test Python Package / integration-tests (push) Successful in 19s
Test Python Package / coverage-report (push) Successful in 7s
Build CI Image / build-and-push (push) Successful in 1m32s
Reviewed-on: LilleVemmelund/python-repositories#49
2026-07-10 14:42:24 +02:00
brian ef31fb4b4e revert 707f11f578
revert chore: release v2.0.4 [skip ci]
2026-07-10 14:42:09 +02:00
CI Bot 707f11f578 chore: release v2.0.4 [skip ci] 2026-07-10 12:36:58 +00:00
brian b1d34f8f0f Merge pull request '[patch] Unify dev tooling via uv so pre-commit, CI, and the update bot stay aligned' (#48) from chore/align-dev-tooling-via-uv into main
Test Python Package / unit-tests (push) Successful in 13s
Code Quality Pipeline / code-quality (push) Successful in 21s
Test Python Package / integration-tests (push) Successful in 20s
Test Python Package / coverage-report (push) Successful in 8s
Build CI Image / build-and-push (push) Successful in 58s
Release on merge to main / release (push) Failing after 9s
Reviewed-on: LilleVemmelund/python-repositories#48
2026-07-10 14:36:44 +02:00
Brian Bjarke JensenandCursor 565879dd52 Unify dev tooling via uv so pre-commit, CI, and the update bot stay aligned.
Test Python Package / unit-tests (pull_request) Successful in 12s
Code Quality Pipeline / code-quality (pull_request) Successful in 19s
PR Title Check / check-title (pull_request) Successful in 31s
Test Python Package / integration-tests (pull_request) Successful in 23s
Test Python Package / coverage-report (pull_request) Successful in 10s
Route Python hooks through uv run with versions pinned in uv.lock, add explicit
Ruff settings, and extend the dependency bot to run pre-commit autoupdate.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-10 14:29:22 +02:00
brian 0d0f430f6f Merge pull request 'Fix automated package publishing in release workflow' (#47) from fix/release-workflow-publish into main
Release on merge to main / release (push) Successful in 5s
Test Python Package / unit-tests (push) Successful in 12s
Code Quality Pipeline / code-quality (push) Successful in 23s
Test Python Package / integration-tests (push) Successful in 19s
Test Python Package / coverage-report (push) Successful in 8s
Reviewed-on: LilleVemmelund/python-repositories#47
2026-07-10 14:17:10 +02:00
Brian Bjarke JensenandCursor 5ac40b1ba5 Publish packages from the release workflow so automated releases reach the registry.
Release commits use [skip ci], which prevents publish.yml from running on tag push.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-10 14:15:04 +02:00
CI Bot fa4ff1f03c chore: release v2.0.3 [skip ci] 2026-07-10 12:06:36 +00:00
brian 0a39151ba0 Merge pull request '[patch] Use structured fields for adapter debug and info logs' (#46) from cursor/structured-adapter-logging into main
Release on merge to main / release (push) Successful in 11s
Code Quality Pipeline / code-quality (push) Successful in 23s
Test Python Package / coverage-report (push) Successful in 8s
Test Python Package / unit-tests (push) Successful in 10s
Test Python Package / integration-tests (push) Successful in 23s
Reviewed-on: LilleVemmelund/python-repositories#46
2026-07-10 14:06:23 +02:00
Brian Bjarke JensenandCursor 82a702db0c Fix mypy error in Minio adapter log assertion.
Test Python Package / unit-tests (pull_request) Successful in 10s
Code Quality Pipeline / code-quality (pull_request) Successful in 33s
Test Python Package / integration-tests (pull_request) Successful in 20s
PR Title Check / check-title (pull_request) Successful in 42s
Test Python Package / coverage-report (pull_request) Successful in 14s
Narrow optional bucket name before checking it appears in captured log text.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-10 14:04:36 +02:00
Brian Bjarke JensenandCursor b886a7c147 Use structured fields for adapter debug and info logs.
PR Title Check / check-title (pull_request) Successful in 6s
Test Python Package / unit-tests (pull_request) Successful in 9s
Code Quality Pipeline / code-quality (pull_request) Failing after 14s
Test Python Package / integration-tests (pull_request) Successful in 1m2s
Test Python Package / coverage-report (pull_request) Successful in 11s
Replace f-string log messages with structlog keyword fields so events aggregate cleanly and Redis payloads are not logged verbatim.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-10 14:00:34 +02:00
brian 4518a93a3a Merge pull request 'Migrate Python workflows to python-repositories-ci runner' (#41) from cursor/migrate-python-repositories-ci into main
Release on merge to main / release (push) Successful in 6s
Test Python Package / unit-tests (push) Successful in 15s
Code Quality Pipeline / code-quality (push) Successful in 28s
Test Python Package / integration-tests (push) Successful in 1m20s
Test Python Package / coverage-report (push) Successful in 11s
Reviewed-on: LilleVemmelund/python-repositories#41
2026-07-10 13:48:49 +02:00
Brian Bjarke JensenandCursor a8b7b59484 Migrate Python workflows to python-repositories-ci runner label.
PR Title Check / check-title (pull_request) Successful in 6s
Test Python Package / unit-tests (pull_request) Successful in 13s
Code Quality Pipeline / code-quality (pull_request) Successful in 23s
Test Python Package / integration-tests (pull_request) Successful in 26s
Test Python Package / coverage-report (pull_request) Successful in 16s
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-10 13:46:10 +02:00
brian 0e6941dccf Merge pull request 'Use lowercase CI image registry owner for Docker compatibility' (#45) from cursor/fix-ci-registry-owner-lowercase into main
Release on merge to main / release (push) Successful in 8s
Test Python Package / integration-tests (push) Successful in 29s
Code Quality Pipeline / code-quality (push) Successful in 48s
Test Python Package / unit-tests (push) Successful in 19s
Test Python Package / coverage-report (push) Successful in 15s
Reviewed-on: LilleVemmelund/python-repositories#45
2026-07-10 13:13:01 +02:00
Brian Bjarke JensenandCursor a6428ccd29 Use lowercase CI image registry owner for Docker compatibility.
PR Title Check / check-title (pull_request) Successful in 7s
Test Python Package / unit-tests (pull_request) Successful in 1m12s
Code Quality Pipeline / code-quality (pull_request) Successful in 1m31s
Test Python Package / integration-tests (pull_request) Successful in 2m25s
Test Python Package / coverage-report (pull_request) Successful in 22s
Docker requires lowercase repository names; Gitea maps lillevemmelund to the LilleVemmelund org.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-10 13:08:28 +02:00
brian 655fdc4ee8 Merge pull request 'Fix CI image registry owner to match Gitea org slug' (#44) from cursor/fix-ci-registry-owner-case into main
Release on merge to main / release (push) Successful in 8s
Test Python Package / unit-tests (push) Successful in 14s
Code Quality Pipeline / code-quality (push) Successful in 30s
Test Python Package / integration-tests (push) Successful in 59s
Test Python Package / coverage-report (push) Successful in 39s
Reviewed-on: LilleVemmelund/python-repositories#44
2026-07-10 13:01:33 +02:00
Brian Bjarke JensenandCursor e031c549e5 Fix CI image registry owner to match Gitea org slug.
PR Title Check / check-title (pull_request) Successful in 7s
Test Python Package / unit-tests (pull_request) Successful in 18s
Test Python Package / integration-tests (pull_request) Successful in 23s
Code Quality Pipeline / code-quality (pull_request) Successful in 34s
Test Python Package / coverage-report (pull_request) Successful in 12s
Use LilleVemmelund instead of lille-vemmelund so container registry pushes resolve the correct namespace.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-10 12:58:29 +02:00
brian 49b6238b7d Merge pull request 'Rename CI registry owner from brian to lille-vemmelund' (#43) from cursor/rename-ci-owner-to-lille-vemmelund into main
Release on merge to main / release (push) Successful in 7s
Code Quality Pipeline / code-quality (push) Successful in 31s
Test Python Package / unit-tests (push) Successful in 37s
Test Python Package / integration-tests (push) Successful in 1m14s
Test Python Package / coverage-report (push) Successful in 13s
Reviewed-on: LilleVemmelund/python-repositories#43
2026-07-10 12:51:03 +02:00
Brian Bjarke JensenandCursor 0549358fe1 Keep python-utils package index under brian owner.
PR Title Check / check-title (pull_request) Successful in 7s
Code Quality Pipeline / code-quality (pull_request) Successful in 29s
Test Python Package / unit-tests (pull_request) Successful in 1m28s
Test Python Package / integration-tests (pull_request) Successful in 1m48s
Test Python Package / coverage-report (pull_request) Successful in 17s
Only the CI image registry owner changes to lille-vemmelund; python-utils remains published under brian.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-10 12:48:32 +02:00
Brian Bjarke JensenandCursor 33efbd1005 Update CI registry owner from brian to lille-vemmelund.
PR Title Check / check-title (pull_request) Successful in 6s
Test Python Package / integration-tests (pull_request) Failing after 30s
Code Quality Pipeline / code-quality (pull_request) Failing after 1m12s
Test Python Package / unit-tests (pull_request) Failing after 1m28s
Test Python Package / coverage-report (pull_request) Has been skipped
Align CI image paths, package index URLs, and lockfile entries with the new Gitea owner name.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-10 09:32:39 +02:00
brian 24126e4044 Merge pull request 'Fix CI image build failures on runners with isolated DinD DNS' (#42) from fix/ci-image-build-dns into main
Release on merge to main / release (push) Successful in 10s
Test Python Package / unit-tests (push) Successful in 16s
Code Quality Pipeline / code-quality (push) Successful in 39s
Build CI Image / build-and-push (push) Failing after 55s
Test Python Package / integration-tests (push) Successful in 1m11s
Test Python Package / coverage-report (push) Successful in 20s
Reviewed-on: brian/python-repositories#42
2026-07-10 09:19:12 +02:00
Brian Bjarke JensenandCursor f3b6cdbc9e Fix CI image build failures on runners with isolated DinD DNS.
PR Title Check / check-title (pull_request) Successful in 8s
Test Python Package / unit-tests (pull_request) Successful in 17s
Code Quality Pipeline / code-quality (pull_request) Successful in 51s
Test Python Package / integration-tests (pull_request) Successful in 1m40s
Test Python Package / coverage-report (pull_request) Successful in 17s
Copy the Docker CLI from the official image instead of apt, and use
--network=host during docker build so apt and layer pulls can resolve
external registries on standalone runners.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-10 09:16:19 +02:00
brian fa2554ef5f Merge pull request 'Add nightly CI base image and migrate workflows to python-repositories-ci' (#40) from cursor/ci-base-image into main
Release on merge to main / release (push) Successful in 7s
Test Python Package / unit-tests (push) Successful in 19s
Code Quality Pipeline / code-quality (push) Successful in 31s
Test Python Package / integration-tests (push) Successful in 1m8s
Test Python Package / coverage-report (push) Successful in 39s
Build CI Image / build-and-push (push) Failing after 6m24s
Reviewed-on: brian/python-repositories#40
2026-07-09 22:51:31 +02:00
Brian Bjarke JensenandCursor b21fd247c5 Revert workflows to ubuntu-latest until CI image is seeded.
PR Title Check / check-title (pull_request) Successful in 6s
Test Python Package / unit-tests (pull_request) Successful in 1m4s
Code Quality Pipeline / code-quality (pull_request) Successful in 1m48s
Test Python Package / integration-tests (pull_request) Successful in 2m10s
Test Python Package / coverage-report (pull_request) Successful in 20s
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-09 22:47:52 +02:00
Brian Bjarke JensenandCursor 441a21204d Add local CI image build script and document bootstrap flow.
Code Quality Pipeline / code-quality (pull_request) Failing after 1s
Test Python Package / unit-tests (pull_request) Failing after 1s
Test Python Package / integration-tests (pull_request) Failing after 1s
Test Python Package / coverage-report (pull_request) Has been skipped
PR Title Check / check-title (pull_request) Successful in 34s
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-09 22:39:33 +02:00
Brian Bjarke JensenandCursor f2a8581de2 Read Python version from .python-version in CI image build.
PR Title Check / check-title (pull_request) Successful in 41s
Code Quality Pipeline / code-quality (pull_request) Failing after 1s
Test Python Package / integration-tests (pull_request) Failing after 34s
Test Python Package / unit-tests (pull_request) Failing after 40s
Test Python Package / coverage-report (pull_request) Has been skipped
Avoid hardcoding 3.12 in the Dockerfile so the CI image tracks the same source of truth as local tooling and uv sync.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-09 22:27:59 +02:00
Brian Bjarke JensenandCursor 3b05260323 Add nightly CI base image and migrate workflows to python-repositories-ci.
Code Quality Pipeline / code-quality (pull_request) Failing after 1s
Test Python Package / unit-tests (pull_request) Failing after 0s
Test Python Package / integration-tests (pull_request) Failing after 0s
PR Title Check / check-title (pull_request) Successful in 6s
Test Python Package / coverage-report (pull_request) Has been skipped
Pre-build Python, uv, and locked deps in a Gitea container image so daily jobs skip repeated bootstrap; document runner registry auth and ci-bot credentials.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-09 22:21:22 +02:00
33 changed files with 538 additions and 149 deletions
+16
View File
@@ -0,0 +1,16 @@
.git
.github
.gitea
.mypy_cache
.pytest_cache
.ruff_cache
.venv
__pycache__
*.pyc
.coverage
coverage-unit
coverage-integration
coverage.txt
dist
build
*.egg-info
+52
View File
@@ -0,0 +1,52 @@
# Build and publish the per-repo CI base image to the Gitea container registry.
name: Build CI Image
on:
schedule:
- cron: "0 2 * * *"
workflow_dispatch:
push:
branches:
- main
paths:
- pyproject.toml
- uv.lock
- docker/ci/**
env:
REGISTRY: gitea.lille-vemmelund.dk
REGISTRY_USER: ci-bot
IMAGE: gitea.lille-vemmelund.dk/lillevemmelund/python-repositories-ci
jobs:
build-and-push:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Log in to Gitea container registry
env:
CI_RUNNER_TOKEN: ${{ secrets.CI_RUNNER_TOKEN }}
run: |
echo "$CI_RUNNER_TOKEN" | docker login "$REGISTRY" -u "$REGISTRY_USER" --password-stdin
- name: Build CI image
env:
CI_RUNNER_TOKEN: ${{ secrets.CI_RUNNER_TOKEN }}
run: |
echo "$CI_RUNNER_TOKEN" > /tmp/uv_token
docker build --network=host -f docker/ci/Dockerfile \
--secret id=uv_token,src=/tmp/uv_token \
-t "${IMAGE}:latest" \
.
rm -f /tmp/uv_token
- name: Tag and push CI image
run: |
STAMP="$(date -u +%Y%m%d%H%M)"
docker tag "${IMAGE}:latest" "${IMAGE}:${STAMP}"
docker push "${IMAGE}:latest"
docker push "${IMAGE}:${STAMP}"
echo "Pushed ${IMAGE}:latest and ${IMAGE}:${STAMP}"
+5 -11
View File
@@ -8,23 +8,17 @@ on:
jobs: jobs:
code-quality: code-quality:
runs-on: ubuntu-latest runs-on: python-repositories-ci
steps: steps:
- name: Checkout code - name: Checkout code
uses: actions/checkout@v4 uses: actions/checkout@v4
- name: Set up Python - name: Sync dependencies
uses: actions/setup-python@v5
with:
python-version-file: .python-version
- name: Install uv
run: pip install uv
- name: Install dependencies
env: env:
UV_LINK_MODE: copy UV_LINK_MODE: copy
run: uv sync --all-extras UV_INDEX_GITEA_USERNAME: ci-bot
UV_INDEX_GITEA_PASSWORD: ${{ secrets.CI_RUNNER_TOKEN }}
run: uv sync --all-extras --frozen
- name: Type check with mypy - name: Type check with mypy
run: uv run mypy . run: uv run mypy .
+29 -8
View File
@@ -7,24 +7,45 @@ on:
jobs: jobs:
update-check: update-check:
runs-on: ubuntu-latest runs-on: python-repositories-ci
steps: steps:
- name: Checkout code - name: Checkout code
uses: actions/checkout@v4 uses: actions/checkout@v4
with: with:
token: ${{ secrets.CI_RUNNER_TOKEN }} token: ${{ secrets.CI_RUNNER_TOKEN }}
- name: Set up Python - name: Sync dependencies
uses: actions/setup-python@v5 env:
with: UV_LINK_MODE: copy
python-version-file: .python-version UV_INDEX_GITEA_USERNAME: ci-bot
UV_INDEX_GITEA_PASSWORD: ${{ secrets.CI_RUNNER_TOKEN }}
- name: Install uv run: uv sync --all-extras
run: pip install uv
- name: Upgrade dependencies - name: Upgrade dependencies
env:
UV_INDEX_GITEA_USERNAME: ci-bot
UV_INDEX_GITEA_PASSWORD: ${{ secrets.CI_RUNNER_TOKEN }}
run: uv lock --upgrade run: uv lock --upgrade
- name: Update pre-commit hook revs
env:
UV_INDEX_GITEA_USERNAME: ci-bot
UV_INDEX_GITEA_PASSWORD: ${{ secrets.CI_RUNNER_TOKEN }}
run: uv run pre-commit autoupdate
- name: Sync upgraded dependencies
env:
UV_LINK_MODE: copy
UV_INDEX_GITEA_USERNAME: ci-bot
UV_INDEX_GITEA_PASSWORD: ${{ secrets.CI_RUNNER_TOKEN }}
run: uv sync --all-extras
- name: Smoke check
run: |
uv run ruff check .
uv run ruff format --check .
uv run mypy .
- name: Commit and push changes - name: Commit and push changes
env: env:
API_URL: ${{ vars.API_URL }} API_URL: ${{ vars.API_URL }}
+9 -10
View File
@@ -9,25 +9,24 @@ on:
jobs: jobs:
build-and-publish: build-and-publish:
runs-on: ubuntu-latest runs-on: python-repositories-ci
steps: steps:
- name: Checkout code - name: Checkout code
uses: actions/checkout@v4 uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version-file: .python-version
- name: Install uv
run: pip install uv
- name: Update version in pyproject.toml to match tag - name: Update version in pyproject.toml to match tag
env:
UV_INDEX_GITEA_USERNAME: ci-bot
UV_INDEX_GITEA_PASSWORD: ${{ secrets.CI_RUNNER_TOKEN }}
run: scripts/ci/bump-version.sh --from-tag "${GITHUB_REF##*/}" run: scripts/ci/bump-version.sh --from-tag "${GITHUB_REF##*/}"
- name: Build package - name: Build package
env:
UV_LINK_MODE: copy
UV_INDEX_GITEA_USERNAME: ci-bot
UV_INDEX_GITEA_PASSWORD: ${{ secrets.CI_RUNNER_TOKEN }}
run: | run: |
uv sync --no-dev uv sync --no-dev --frozen
uv build uv build
- name: Publish to Gitea Package Registry - name: Publish to Gitea Package Registry
+24 -11
View File
@@ -8,7 +8,7 @@ on:
jobs: jobs:
release: release:
if: ${{ !contains(github.event.head_commit.message, '[skip ci]') }} if: ${{ !contains(github.event.head_commit.message, '[skip ci]') }}
runs-on: ubuntu-latest runs-on: python-repositories-ci
steps: steps:
- name: Checkout code - name: Checkout code
uses: actions/checkout@v4 uses: actions/checkout@v4
@@ -22,19 +22,12 @@ jobs:
COMMIT_MSG: ${{ github.event.head_commit.message }} COMMIT_MSG: ${{ github.event.head_commit.message }}
run: scripts/ci/parse-merge-commit.sh "$COMMIT_MSG" run: scripts/ci/parse-merge-commit.sh "$COMMIT_MSG"
- name: Set up Python
if: steps.meta.outputs.bump != 'skip'
uses: actions/setup-python@v5
with:
python-version-file: .python-version
- name: Install uv
if: steps.meta.outputs.bump != 'skip'
run: pip install uv
- name: Bump version - name: Bump version
if: steps.meta.outputs.bump != 'skip' if: steps.meta.outputs.bump != 'skip'
id: bump id: bump
env:
UV_INDEX_GITEA_USERNAME: ci-bot
UV_INDEX_GITEA_PASSWORD: ${{ secrets.CI_RUNNER_TOKEN }}
run: scripts/ci/bump-version.sh "${{ steps.meta.outputs.bump }}" run: scripts/ci/bump-version.sh "${{ steps.meta.outputs.bump }}"
- name: Generate release notes - name: Generate release notes
@@ -81,3 +74,23 @@ jobs:
-H "Authorization: token ${CI_RUNNER_TOKEN}" \ -H "Authorization: token ${CI_RUNNER_TOKEN}" \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d "$PAYLOAD" -d "$PAYLOAD"
# Publish here instead of relying on publish.yml tag trigger: release commits use
# [skip ci], which suppresses tag-push workflows, and Gitea does not emit release
# events for releases created by the CI bot account.
- name: Build package
if: steps.meta.outputs.bump != 'skip'
env:
UV_LINK_MODE: copy
UV_INDEX_GITEA_USERNAME: ci-bot
UV_INDEX_GITEA_PASSWORD: ${{ secrets.CI_RUNNER_TOKEN }}
run: |
uv sync --no-dev --frozen
uv build
- name: Publish to Gitea Package Registry
if: steps.meta.outputs.bump != 'skip'
env:
UV_PUBLISH_TOKEN: ${{ secrets.CI_RUNNER_TOKEN }}
UV_PUBLISH_URL: ${{ vars.REPOSITORY_URL }}
run: uv publish
+5 -11
View File
@@ -7,23 +7,17 @@ on:
jobs: jobs:
safety: safety:
runs-on: ubuntu-latest runs-on: python-repositories-ci
steps: steps:
- name: Checkout code - name: Checkout code
uses: actions/checkout@v4 uses: actions/checkout@v4
- name: Set up Python - name: Sync dependencies
uses: actions/setup-python@v5
with:
python-version-file: .python-version
- name: Install uv
run: pip install uv
- name: Install dependencies
env: env:
UV_LINK_MODE: copy UV_LINK_MODE: copy
run: uv sync UV_INDEX_GITEA_USERNAME: ci-bot
UV_INDEX_GITEA_PASSWORD: ${{ secrets.CI_RUNNER_TOKEN }}
run: uv sync --all-extras --frozen
- name: Run safety check - name: Run safety check
run: uv run safety check run: uv run safety check
+15 -33
View File
@@ -8,23 +8,17 @@ on:
jobs: jobs:
unit-tests: unit-tests:
runs-on: ubuntu-latest runs-on: python-repositories-ci
steps: steps:
- name: Checkout code - name: Checkout code
uses: actions/checkout@v4 uses: actions/checkout@v4
- name: Set up Python - name: Sync dependencies
uses: actions/setup-python@v5
with:
python-version-file: .python-version
- name: Install uv
run: pip install uv
- name: Install dependencies
env: env:
UV_LINK_MODE: copy UV_LINK_MODE: copy
run: uv sync --all-extras UV_INDEX_GITEA_USERNAME: ci-bot
UV_INDEX_GITEA_PASSWORD: ${{ secrets.CI_RUNNER_TOKEN }}
run: uv sync --all-extras --frozen
- name: Run unit tests - name: Run unit tests
run: | run: |
@@ -43,23 +37,17 @@ jobs:
compression-level: 0 compression-level: 0
integration-tests: integration-tests:
runs-on: ubuntu-latest runs-on: python-repositories-ci
steps: steps:
- name: Checkout code - name: Checkout code
uses: actions/checkout@v4 uses: actions/checkout@v4
- name: Set up Python - name: Sync dependencies
uses: actions/setup-python@v5
with:
python-version-file: .python-version
- name: Install uv
run: pip install uv
- name: Install dependencies
env: env:
UV_LINK_MODE: copy UV_LINK_MODE: copy
run: uv sync --all-extras UV_INDEX_GITEA_USERNAME: ci-bot
UV_INDEX_GITEA_PASSWORD: ${{ secrets.CI_RUNNER_TOKEN }}
run: uv sync --all-extras --frozen
- name: Verify Docker - name: Verify Docker
run: docker info run: docker info
@@ -83,24 +71,18 @@ jobs:
coverage-report: coverage-report:
# Merges unit + integration coverage and enforces fail_under from pyproject.toml. # Merges unit + integration coverage and enforces fail_under from pyproject.toml.
needs: [unit-tests, integration-tests] needs: [unit-tests, integration-tests]
runs-on: ubuntu-latest runs-on: python-repositories-ci
if: github.event_name == 'pull_request' || github.event_name == 'push' if: github.event_name == 'pull_request' || github.event_name == 'push'
steps: steps:
- name: Checkout code - name: Checkout code
uses: actions/checkout@v4 uses: actions/checkout@v4
- name: Set up Python - name: Sync dependencies
uses: actions/setup-python@v5
with:
python-version-file: .python-version
- name: Install uv
run: pip install uv
- name: Install dependencies
env: env:
UV_LINK_MODE: copy UV_LINK_MODE: copy
run: uv sync --all-extras UV_INDEX_GITEA_USERNAME: ci-bot
UV_INDEX_GITEA_PASSWORD: ${{ secrets.CI_RUNNER_TOKEN }}
run: uv sync --all-extras --frozen
- name: Download unit coverage - name: Download unit coverage
uses: https://github.com/christopherHX/gitea-download-artifact@v4 uses: https://github.com/christopherHX/gitea-download-artifact@v4
+19 -17
View File
@@ -11,28 +11,30 @@ repos:
- id: name-tests-test - id: name-tests-test
- id: check-merge-conflict - id: check-merge-conflict
# Python linting and formatting with Ruff # Python tooling via uv (versions pinned in uv.lock)
- repo: https://github.com/astral-sh/ruff-pre-commit - repo: local
rev: v0.12.8
hooks: hooks:
- id: ruff-check - id: ruff-check
args: [--fix] name: ruff check
entry: uv run ruff check --fix
language: system
types: [python]
- id: ruff-format - id: ruff-format
name: ruff format
# Static type checking with mypy entry: uv run ruff format
- repo: https://github.com/pre-commit/mirrors-mypy language: system
rev: v1.8.0 types: [python]
hooks:
- id: mypy - id: mypy
additional_dependencies: name: mypy
- types-redis entry: uv run mypy .
language: system
# Python syntax modernization with pyupgrade types: [python]
- repo: https://github.com/asottile/pyupgrade pass_filenames: false
rev: v3.20.0
hooks:
- id: pyupgrade - id: pyupgrade
args: ["--py312-plus"] name: pyupgrade
entry: uv run pyupgrade --py312-plus
language: system
types: [python]
# Formatting for Markdown, JSON, and YAML with Prettier # Formatting for Markdown, JSON, and YAML with Prettier
- repo: https://github.com/pre-commit/mirrors-prettier - repo: https://github.com/pre-commit/mirrors-prettier
+10 -3
View File
@@ -166,7 +166,14 @@ uv run pytest -v # full suite (requires Doc
Integration tests are marked with `@pytest.mark.integration` and require Docker (testcontainers). Run unit tests alone for quick local feedback. Integration tests are marked with `@pytest.mark.integration` and require Docker (testcontainers). Run unit tests alone for quick local feedback.
CI runs unit and integration tests in parallel with coverage, then merges `.coverage` artifacts in a follow-up job (via [christopherhx/gitea-\*-artifact@v4](https://github.com/christopherHX/gitea-upload-artifact) for Gitea 1.26 compatibility). Combined coverage must be at least **90%**; the floor is set by [`fail_under` in `pyproject.toml`](pyproject.toml#L45-L48) and enforced after merging unit and integration coverage, not on unit-only runs. ### CI base image
Gitea Actions jobs use a pre-built image (`python-repositories-ci`) with Python 3.12,
`uv`, and locked dependencies baked in. The image is rebuilt nightly and when
`uv.lock` changes; see [`docs/ci-image.md`](docs/ci-image.md) for runner setup and
bootstrap order.
CI runs unit and integration tests in parallel with coverage, then merges `.coverage` artifacts in a follow-up job (via [christopherhx/gitea-\*-artifact@v4](https://github.com/christopherHX/gitea-upload-artifact) for Gitea 1.26 compatibility). Combined coverage must meet the floor in [`fail_under` in `pyproject.toml`](pyproject.toml#L45-L48); enforcement happens after merging unit and integration coverage, not on unit-only runs.
To check coverage locally (requires Docker for the full suite): To check coverage locally (requires Docker for the full suite):
@@ -191,8 +198,8 @@ Releases are automated when a pull request is merged to `main`. CI reads the **m
1. Open a PR targeting `main` (see [`.gitea/PULL_REQUEST_TEMPLATE.md`](.gitea/PULL_REQUEST_TEMPLATE.md)). 1. Open a PR targeting `main` (see [`.gitea/PULL_REQUEST_TEMPLATE.md`](.gitea/PULL_REQUEST_TEMPLATE.md)).
2. If the PR changes files under `python_repositories/`, the title **must** start with a version bump prefix (enforced by CI). 2. If the PR changes files under `python_repositories/`, the title **must** start with a version bump prefix (enforced by CI).
3. On merge, [`release.yml`](.gitea/workflows/release.yml) bumps [`pyproject.toml`](pyproject.toml) and syncs [`uv.lock`](uv.lock), commits, tags `vX.Y.Z`, creates a Gitea release with auto-generated notes, and pushes the tag. 3. On merge, [`release.yml`](.gitea/workflows/release.yml) bumps [`pyproject.toml`](pyproject.toml) and syncs [`uv.lock`](uv.lock), commits, tags `vX.Y.Z`, creates a Gitea release with auto-generated notes, pushes the tag, and publishes the package to the Gitea Package Registry.
4. [`publish.yml`](.gitea/workflows/publish.yml) builds and publishes the package to the Gitea Package Registry. 4. [`publish.yml`](.gitea/workflows/publish.yml) handles manual `v*.*.*` tag pushes only (automated releases publish from `release.yml` because release commits use `[skip ci]`, which suppresses tag-push workflows).
Docs-, CI-, and test-only PRs do not need a prefix and will not trigger a release. Docs-, CI-, and test-only PRs do not need a prefix and will not trigger a release.
+29
View File
@@ -0,0 +1,29 @@
# CI job image for Gitea Actions (act_runner requires Node.js in job containers).
FROM node:20-bookworm
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
ca-certificates \
curl \
git \
&& rm -rf /var/lib/apt/lists/*
COPY --from=docker:27-cli /usr/local/bin/docker /usr/local/bin/docker
COPY --from=ghcr.io/astral-sh/uv:0.7.0 /uv /usr/local/bin/uv
ENV UV_PYTHON_INSTALL_DIR=/opt/uv-python \
UV_LINK_MODE=copy
COPY .python-version /tmp/.python-version
RUN uv python install "$(cat /tmp/.python-version)"
WORKDIR /app
COPY pyproject.toml uv.lock .python-version ./
ENV PATH="/app/.venv/bin:${PATH}"
RUN --mount=type=secret,id=uv_token \
UV_INDEX_GITEA_USERNAME=ci-bot \
UV_INDEX_GITEA_PASSWORD="$(cat /run/secrets/uv_token)" \
uv sync --all-extras --frozen --no-install-project
+169
View File
@@ -0,0 +1,169 @@
# CI base image
This repository uses a **per-repo Docker image** for Gitea Actions jobs instead of
installing Python and `uv` on every run. Infrastructure images (redis, minio, etc.)
are cached cluster-wide via Harbor (see homelab-platform
[`docs/harbor-registry-mirror.md`](https://gitea.lille-vemmelund.dk/LilleVemmelund/homelab-platform/src/branch/main/docs/harbor-registry-mirror.md)).
## Image contents
Built from [`docker/ci/Dockerfile`](../docker/ci/Dockerfile):
- Node.js 20 (required by act_runner job containers)
- Python 3.12 (installed via `uv python install` from [`.python-version`](../.python-version)) and pinned `uv` 0.7.0
- Docker CLI (integration tests via testcontainers)
- Dev dependencies from `uv.lock` (`uv sync --all-extras --no-install-project`)
Published to the Gitea container registry (owner segment must be lowercase for Docker):
- `gitea.lille-vemmelund.dk/lillevemmelund/python-repositories-ci:latest`
- `gitea.lille-vemmelund.dk/lillevemmelund/python-repositories-ci:YYYYMMDDHHmm` (timestamped rollback tag)
## Rebuild triggers
[`ci-image.yml`](../.gitea/workflows/ci-image.yml) runs on:
- Nightly cron (`0 2 * * *` UTC)
- Manual `workflow_dispatch`
- Push to `main` when `pyproject.toml`, `uv.lock`, or `docker/ci/**` change
## Workflow usage
Python jobs use `runs-on: python-repositories-ci` and a fast incremental sync:
```yaml
runs-on: python-repositories-ci
steps:
- uses: actions/checkout@v4
- name: Sync dependencies
env:
UV_LINK_MODE: copy
UV_INDEX_GITEA_USERNAME: ci-bot
UV_INDEX_GITEA_PASSWORD: ${{ secrets.CI_RUNNER_TOKEN }}
run: uv sync --all-extras --frozen
```
`ci-image.yml` uses `runs-on: ubuntu-latest` so it can bootstrap before the custom
image exists.
## Registry authentication
act_runner pulls the job image **before any workflow step runs**, so a `docker
login` step inside a job cannot authenticate that pull. Authentication must be
configured on the runner (or via a job-level `container.credentials` block — not
used here).
### k8s Gitea runners (automatic)
The homelab-platform runners mount Gitea registry credentials automatically:
- Secret: `gitea-runners/gitea-registry-dockerconfig` (created by
[`scripts/create-gitea-registry-secret.sh`](https://gitea.lille-vemmelund.dk/LilleVemmelund/homelab-platform/src/branch/main/scripts/create-gitea-registry-secret.sh))
- Mounted at `/root/.docker/config.json` on the **runner** container (not DinD)
- Configured in
[`platform/gitea-runners/values.yaml`](https://gitea.lille-vemmelund.dk/LilleVemmelund/homelab-platform/src/branch/main/platform/gitea-runners/values.yaml)
No manual `docker login` is required on the in-cluster runners once the secret
exists. To create or rotate credentials:
```bash
export KUBECONFIG=/path/to/homelab-cluster/talos/_out/kubeconfig
export GITEA_REGISTRY_USERNAME='ci-bot'
export GITEA_REGISTRY_PASSWORD='personal-access-token-with-read-package'
bash scripts/create-gitea-registry-secret.sh
```
Then restart runner pods so they pick up the updated secret. See homelab-platform
[`docs/gitea-actions-runners.md`](https://gitea.lille-vemmelund.dk/LilleVemmelund/homelab-platform/src/branch/main/docs/gitea-actions-runners.md)
for full runner setup.
Verify on a running pod:
```bash
kubectl -n gitea-runners exec homelab-cluster-gitea-runner-0 -c runner -- \
test -f /root/.docker/config.json && echo "registry auth mounted"
```
### Standalone runners (e.g. Unraid `homelab`)
The 4th runner is outside the k8s cluster and does **not** get the automatic
mount. Configure registry auth manually on that host:
```bash
docker login gitea.lille-vemmelund.dk -u ci-bot -p <token>
```
Also add the `python-repositories-ci` label to that runner's act_runner config.
### `ci-image.yml` push login
The build workflow still runs `docker login` before `docker push`. That step
authenticates **DinD inside the job** for pushing the image to Gitea — a different
code path from act_runner pulling the job container.
## Runner label
Jobs use the `python-repositories-ci` act_runner label, configured in
homelab-platform
[`platform/gitea-runners/values.yaml`](https://gitea.lille-vemmelund.dk/LilleVemmelund/homelab-platform/src/branch/main/platform/gitea-runners/values.yaml):
```yaml
python-repositories-ci:docker://gitea.lille-vemmelund.dk/lillevemmelund/python-repositories-ci:latest
```
After label changes, roll runner pods so they re-register with Gitea.
## Bootstrap order
1. Ensure homelab-platform runners have `gitea-registry-dockerconfig` and the
`python-repositories-ci` label (see homelab-platform docs).
2. Merge `ci-image.yml`, `docker/ci/Dockerfile`, and workflow migrations to `main`.
3. Seed the registry with a first image (see below).
4. Confirm a test workflow job starts on `python-repositories-ci`.
Until step 3 completes, jobs targeting `python-repositories-ci` will fail because
the image does not exist in the Gitea registry yet.
### Seed the image
**After merge to `main`:** Actions → **Build CI Image****Run workflow**.
**Before merge (e.g. PR branch):** the workflow file is not on `main` yet — build
and push locally with [`scripts/ci/build-ci-image.sh`](../scripts/ci/build-ci-image.sh):
```bash
export CI_RUNNER_TOKEN='ci-bot-personal-access-token'
bash scripts/ci/build-ci-image.sh --push
```
Run from the repo root on the branch you want to test. Runners pull
`gitea.lille-vemmelund.dk/lillevemmelund/python-repositories-ci:latest` from the registry;
they do not care which git branch built it.
Override registry settings if needed:
```bash
export REGISTRY=gitea.lille-vemmelund.dk
export REGISTRY_USER=ci-bot
export CI_RUNNER_TOKEN='...'
bash scripts/ci/build-ci-image.sh --push
```
## Local build
Build only (no registry login or push):
```bash
export CI_RUNNER_TOKEN='ci-bot-personal-access-token'
bash scripts/ci/build-ci-image.sh --local
```
Or manually:
```bash
echo "$CI_RUNNER_TOKEN" > /tmp/uv_token
docker build -f docker/ci/Dockerfile \
--secret id=uv_token,src=/tmp/uv_token \
-t python-repositories-ci:local .
rm -f /tmp/uv_token
```
+17 -5
View File
@@ -1,6 +1,6 @@
[project] [project]
name = "python-repositories" name = "python-repositories"
version = "2.0.2" version = "2.0.4"
description = "Various python repository interfaces exposed as a python package." description = "Various python repository interfaces exposed as a python package."
authors = [ authors = [
{ name = "Brian Bjarke Jensen", email = "schnitzelen@gmail.com" } { name = "Brian Bjarke Jensen", email = "schnitzelen@gmail.com" }
@@ -60,6 +60,18 @@ name = "gitea"
url = "https://gitea.lille-vemmelund.dk/api/packages/brian/pypi/simple/" url = "https://gitea.lille-vemmelund.dk/api/packages/brian/pypi/simple/"
explicit = true explicit = true
[tool.ruff]
line-length = 88
target-version = "py312"
[tool.ruff.lint]
extend-select = ["I"]
[tool.ruff.lint.isort]
known-first-party = ["python_repositories", "tests"]
combine-as-imports = true
force-sort-within-sections = true
[tool.mypy] [tool.mypy]
python_version = "3.12" python_version = "3.12"
warn_return_any = true # nudge to use stricter types warn_return_any = true # nudge to use stricter types
@@ -87,12 +99,12 @@ ignore_missing_imports = true
[dependency-groups] [dependency-groups]
dev = [ dev = [
"mypy>=1.17.1", "mypy>=2.1.0",
"pre-commit>=4.3.0", "pre-commit>=4.6.0",
"pytest>=8.4.1", "pytest>=8.4.1",
"pytest-cov>=7.0.0", "pytest-cov>=7.0.0",
"pyupgrade>=3.20.0", "pyupgrade>=3.21.2",
"ruff>=0.12.11", "ruff>=0.15.20",
"safety>=3.6.0", "safety>=3.6.0",
"testcontainers>=4.13.0", "testcontainers>=4.13.0",
"types-redis>=4.6.0.20241004", "types-redis>=4.6.0.20241004",
@@ -2,8 +2,8 @@
from __future__ import annotations from __future__ import annotations
import time
from abc import abstractmethod from abc import abstractmethod
import time
from typing import Self from typing import Self
import structlog import structlog
@@ -73,7 +73,7 @@ class ConnectionAwareAdapter(ConnectionAwareInterface, ContextAwareInterface):
self._invalidate_health_cache() self._invalidate_health_cache()
return return
if self._is_client_ready() and self.is_connected(): if self._is_client_ready() and self.is_connected():
self.logger.info(f"Already connected to {self.connection_name}") self.logger.info("Already connected", connection_name=self.connection_name)
return return
self.disconnect() self.disconnect()
self._establish_connection() self._establish_connection()
+12 -6
View File
@@ -1,6 +1,7 @@
"""Definition of MinioAdapter class.""" """Definition of MinioAdapter class."""
from __future__ import annotations from __future__ import annotations
from io import BytesIO from io import BytesIO
from python_repositories.adapters.connection_aware_adapter import ( from python_repositories.adapters.connection_aware_adapter import (
@@ -88,7 +89,7 @@ class MinioAdapter(ObjectRepositoryInterface, ConnectionAwareAdapter):
raise ConnectionError( raise ConnectionError(
f"Bucket '{bucket}' does not exist on Minio at {endpoint}" f"Bucket '{bucket}' does not exist on Minio at {endpoint}"
) )
self.logger.info(f"Creating bucket '{bucket}'") self.logger.info("Creating bucket", bucket=bucket)
client.make_bucket(bucket) client.make_bucket(bucket)
self._client = client self._client = client
self._bucket_name = bucket self._bucket_name = bucket
@@ -138,7 +139,7 @@ class MinioAdapter(ObjectRepositoryInterface, ConnectionAwareAdapter):
content_type=content_type, content_type=content_type,
) )
self.logger.debug( self.logger.debug(
f"Put object '{object_name}' into bucket '{self._bucket_name}'" "Put object", object_name=object_name, bucket=self._bucket_name
) )
def get(self, object_name: str) -> BytesIO | None: def get(self, object_name: str) -> BytesIO | None:
@@ -163,13 +164,15 @@ class MinioAdapter(ObjectRepositoryInterface, ConnectionAwareAdapter):
buffer.write(chunk) buffer.write(chunk)
buffer.seek(0) buffer.seek(0)
self.logger.debug( self.logger.debug(
f"Got object '{object_name}' from bucket '{self._bucket_name}'" "Got object", object_name=object_name, bucket=self._bucket_name
) )
return buffer return buffer
except minio.S3Error as exc: except minio.S3Error as exc:
if exc.code == "NoSuchKey": if exc.code == "NoSuchKey":
self.logger.warning( self.logger.warning(
f"Object '{object_name}' not found in bucket '{self._bucket_name}'" "Object not found",
object_name=object_name,
bucket=self._bucket_name,
) )
return None return None
raise raise
@@ -193,7 +196,7 @@ class MinioAdapter(ObjectRepositoryInterface, ConnectionAwareAdapter):
object_name=object_name, object_name=object_name,
) )
self.logger.debug( self.logger.debug(
f"Deleted object '{object_name}' from bucket '{self._bucket_name}'" "Deleted object", object_name=object_name, bucket=self._bucket_name
) )
def list_objects(self, prefix: str = "") -> list[str]: def list_objects(self, prefix: str = "") -> list[str]:
@@ -215,6 +218,9 @@ class MinioAdapter(ObjectRepositoryInterface, ConnectionAwareAdapter):
obj.object_name for obj in objects if obj.object_name is not None obj.object_name for obj in objects if obj.object_name is not None
] ]
self.logger.debug( self.logger.debug(
f"Listed {len(object_names)} object(s) in bucket '{self._bucket_name}' with prefix '{prefix}'" "Listed objects",
count=len(object_names),
bucket=self._bucket_name,
prefix=prefix,
) )
return object_names return object_names
@@ -1,6 +1,7 @@
"""Definition of RedisAdapter class.""" """Definition of RedisAdapter class."""
from __future__ import annotations from __future__ import annotations
from collections.abc import Iterator from collections.abc import Iterator
from typing import Any, cast from typing import Any, cast
@@ -101,7 +102,7 @@ class RedisAdapter(JsonRepositoryInterface, ConnectionAwareAdapter):
assert self._client is not None assert self._client is not None
# Set data # Set data
self._client.json().set(key, self.path, data) self._client.json().set(key, self.path, data)
self.logger.debug(f"Set {key} to {data}") self.logger.debug("Set key", key=key, data_keys=list(data.keys()))
def get(self, key: str) -> dict[str, Any] | None: def get(self, key: str) -> dict[str, Any] | None:
"""Get a JSON object from Redis.""" """Get a JSON object from Redis."""
@@ -116,7 +117,7 @@ class RedisAdapter(JsonRepositoryInterface, ConnectionAwareAdapter):
dict[str, Any] | None, dict[str, Any] | None,
self._client.json().get(key), self._client.json().get(key),
) )
self.logger.debug(f"Got {data} from {key}") self.logger.debug("Got value", key=key, found=data is not None)
return data return data
def delete(self, key: str) -> None: def delete(self, key: str) -> None:
@@ -129,7 +130,7 @@ class RedisAdapter(JsonRepositoryInterface, ConnectionAwareAdapter):
assert self._client is not None assert self._client is not None
# Delete data # Delete data
self._client.json().delete(key) self._client.json().delete(key)
self.logger.debug(f"Deleted {key}") self.logger.debug("Deleted key", key=key)
def _validate_pattern(self, pattern: str) -> None: def _validate_pattern(self, pattern: str) -> None:
if not isinstance(pattern, str) or len(pattern) == 0: if not isinstance(pattern, str) or len(pattern) == 0:
@@ -147,7 +148,7 @@ class RedisAdapter(JsonRepositoryInterface, ConnectionAwareAdapter):
self._client.keys(pattern), self._client.keys(pattern),
) )
keys: list[str] = [key.decode(self.encoding) for key in keys_raw] keys: list[str] = [key.decode(self.encoding) for key in keys_raw]
self.logger.debug(f"Got {keys} matching {pattern}") self.logger.debug("Listed keys", pattern=pattern, count=len(keys))
return keys return keys
def scan_keys( def scan_keys(
+1 -1
View File
@@ -2,8 +2,8 @@
from __future__ import annotations from __future__ import annotations
import os
from dataclasses import dataclass from dataclasses import dataclass
import os
from python_utils import check_env from python_utils import check_env
+1 -1
View File
@@ -2,8 +2,8 @@
from __future__ import annotations from __future__ import annotations
import os
from dataclasses import dataclass from dataclasses import dataclass
import os
from python_utils import check_env from python_utils import check_env
@@ -1,7 +1,7 @@
"""Definition of JsonRepositoryInterface abstract base class.""" """Definition of JsonRepositoryInterface abstract base class."""
from collections.abc import Iterator
from abc import ABC, abstractmethod from abc import ABC, abstractmethod
from collections.abc import Iterator
from typing import Any from typing import Any
+78
View File
@@ -0,0 +1,78 @@
#!/usr/bin/env bash
# Build and optionally push the Gitea Actions CI base image locally.
#
# Use when ci-image.yml is not yet on main (e.g. bootstrapping a PR branch) or
# when you need to rebuild without waiting for the nightly workflow.
#
# Build only (smoke test):
# export CI_RUNNER_TOKEN='ci-bot-personal-access-token'
# bash scripts/ci/build-ci-image.sh --local
#
# Build and push to Gitea (unblocks python-repositories-ci jobs):
# export CI_RUNNER_TOKEN='ci-bot-personal-access-token'
# bash scripts/ci/build-ci-image.sh --push
#
# Optional overrides:
# REGISTRY=gitea.lille-vemmelund.dk
# REGISTRY_USER=ci-bot
# IMAGE_OWNER=lillevemmelund
# IMAGE_NAME=python-repositories-ci
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
REPO_ROOT="$(cd "${SCRIPT_DIR}/../.." && pwd)"
REGISTRY="${REGISTRY:-gitea.lille-vemmelund.dk}"
REGISTRY_USER="${REGISTRY_USER:-ci-bot}"
IMAGE_OWNER="${IMAGE_OWNER:-lillevemmelund}"
IMAGE_NAME="${IMAGE_NAME:-python-repositories-ci}"
IMAGE="${IMAGE:-${REGISTRY}/${IMAGE_OWNER}/${IMAGE_NAME}}"
MODE="push"
if [[ "${1:-}" == "--local" ]]; then
MODE="local"
elif [[ "${1:-}" == "--push" || -z "${1:-}" ]]; then
MODE="push"
elif [[ -n "${1:-}" ]]; then
echo "Usage: build-ci-image.sh [--local | --push]" >&2
exit 1
fi
: "${CI_RUNNER_TOKEN:?CI_RUNNER_TOKEN is required (ci-bot token with read/write package access)}"
TOKEN_FILE="$(mktemp)"
cleanup() {
rm -f "$TOKEN_FILE"
}
trap cleanup EXIT
printf '%s' "$CI_RUNNER_TOKEN" >"$TOKEN_FILE"
cd "$REPO_ROOT"
if [[ "$MODE" == "local" ]]; then
echo "=== Building local CI image (no push): python-repositories-ci:local ==="
docker build --network=host -f docker/ci/Dockerfile \
--secret "id=uv_token,src=${TOKEN_FILE}" \
-t python-repositories-ci:local \
.
echo "Built python-repositories-ci:local"
exit 0
fi
echo "=== Logging in to ${REGISTRY} as ${REGISTRY_USER} ==="
echo "$CI_RUNNER_TOKEN" | docker login "$REGISTRY" -u "$REGISTRY_USER" --password-stdin
echo "=== Building ${IMAGE}:latest ==="
docker build --network=host -f docker/ci/Dockerfile \
--secret "id=uv_token,src=${TOKEN_FILE}" \
-t "${IMAGE}:latest" \
.
STAMP="$(date -u +%Y%m%d%H%M)"
echo "=== Pushing ${IMAGE}:latest and ${IMAGE}:${STAMP} ==="
docker tag "${IMAGE}:latest" "${IMAGE}:${STAMP}"
docker push "${IMAGE}:latest"
docker push "${IMAGE}:${STAMP}"
echo "Done. Re-run failing CI jobs — runners pull ${IMAGE}:latest"
+9 -3
View File
@@ -11,15 +11,16 @@ set -euo pipefail
BRANCH="${BRANCH:-renovate/auto-deps-update}" BRANCH="${BRANCH:-renovate/auto-deps-update}"
BASE_BRANCH="${BASE_BRANCH:-main}" BASE_BRANCH="${BASE_BRANCH:-main}"
LOCKFILE="${LOCKFILE:-uv.lock}" LOCKFILE="${LOCKFILE:-uv.lock}"
PRE_COMMIT_CONFIG="${PRE_COMMIT_CONFIG:-.pre-commit-config.yaml}"
if git diff --quiet "$LOCKFILE"; then if git diff --quiet "$LOCKFILE" && git diff --quiet "$PRE_COMMIT_CONFIG"; then
echo "No dependency updates available." echo "No dependency updates available."
exit 0 exit 0
fi fi
git config user.name "CI Bot" git config user.name "CI Bot"
git config user.email "ci-bot@example.com" git config user.email "ci-bot@example.com"
git add "$LOCKFILE" git add "$LOCKFILE" "$PRE_COMMIT_CONFIG"
git commit -m "chore(deps): update dependencies [automated]" git commit -m "chore(deps): update dependencies [automated]"
git remote set-url origin "https://x-access-token:${CI_RUNNER_TOKEN}@${GITHUB_SERVER_URL#https://}/${GITHUB_REPOSITORY}.git" git remote set-url origin "https://x-access-token:${CI_RUNNER_TOKEN}@${GITHUB_SERVER_URL#https://}/${GITHUB_REPOSITORY}.git"
@@ -48,7 +49,12 @@ print(
json.dumps( json.dumps(
{ {
"title": "chore(deps): update dependencies", "title": "chore(deps): update dependencies",
"body": "This PR was created automatically to update dependencies.", "body": (
"This PR was created automatically to update dependencies.\n\n"
"- `uv.lock`: upgraded Python dependencies\n"
"- `.pre-commit-config.yaml`: updated remote hook revs via "
"`pre-commit autoupdate`"
),
"head": os.environ["BRANCH"], "head": os.environ["BRANCH"],
"base": os.environ["BASE_BRANCH"], "base": os.environ["BASE_BRANCH"],
} }
+2 -2
View File
@@ -1,13 +1,13 @@
"""Integration tests configuration.""" """Integration tests configuration."""
import logging
from collections.abc import Generator from collections.abc import Generator
import logging
from typing import Any, cast from typing import Any, cast
from minio import Minio
import pytest import pytest
import redis import redis
import structlog import structlog
from minio import Minio
from testcontainers.core.container import DockerContainer from testcontainers.core.container import DockerContainer
from testcontainers.core.waiting_utils import WaitStrategy, WaitStrategyTarget from testcontainers.core.waiting_utils import WaitStrategy, WaitStrategyTarget
from testcontainers.minio import MinioContainer from testcontainers.minio import MinioContainer
+1 -1
View File
@@ -5,8 +5,8 @@ from io import BytesIO
import os import os
import random import random
import pytest
from minio import Minio from minio import Minio
import pytest
from python_repositories.examples.artifact_object_repository import ( from python_repositories.examples.artifact_object_repository import (
ArtifactObjectRepository, ArtifactObjectRepository,
+11 -8
View File
@@ -2,13 +2,13 @@
from collections.abc import Generator from collections.abc import Generator
from dataclasses import replace from dataclasses import replace
from io import BytesIO
import logging import logging
import random import random
from io import BytesIO
from unittest.mock import MagicMock from unittest.mock import MagicMock
import pytest
from minio import Minio, S3Error from minio import Minio, S3Error
import pytest
from urllib3.response import BaseHTTPResponse from urllib3.response import BaseHTTPResponse
from python_repositories.adapters.minio_adapter import MinioAdapter from python_repositories.adapters.minio_adapter import MinioAdapter
@@ -102,7 +102,8 @@ def test_should_log_info_when_already_connected(
"""Test that the MinioAdapter logs info when connect is called while already connected.""" """Test that the MinioAdapter logs info when connect is called while already connected."""
with caplog.at_level(logging.INFO): with caplog.at_level(logging.INFO):
minio_adapter.connect() minio_adapter.connect()
assert "Already connected to Minio" in caplog.text assert "Already connected" in caplog.text
assert "Minio" in caplog.text
def test_should_raise_connection_error_when_unable_to_connect() -> None: def test_should_raise_connection_error_when_unable_to_connect() -> None:
@@ -150,7 +151,8 @@ def test_connect_creates_bucket_when_create_bucket_if_missing_enabled(
with caplog.at_level(logging.INFO): with caplog.at_level(logging.INFO):
adapter.connect() adapter.connect()
assert f"Creating bucket '{bucket_name}'" in caplog.text assert "Creating bucket" in caplog.text
assert bucket_name in caplog.text
def test_should_log_error_on_exception_during_exit( def test_should_log_error_on_exception_during_exit(
@@ -233,10 +235,11 @@ def test_should_log_warning_when_getting_nonexistent_object(
with caplog.at_level("WARNING"): with caplog.at_level("WARNING"):
result = adapter.get(object_name) result = adapter.get(object_name)
assert result is None assert result is None
assert ( assert "Object not found" in caplog.text
f"Object '{object_name}' not found in bucket '{adapter._bucket_name}'" assert object_name in caplog.text
in caplog.text bucket_name = adapter._bucket_name
) assert bucket_name is not None
assert bucket_name in caplog.text
def test_should_reraise_s3error_other_than_no_such_key() -> None: def test_should_reraise_s3error_other_than_no_such_key() -> None:
+2 -1
View File
@@ -56,7 +56,8 @@ def test_should_log_info_when_already_connected(
"""Test that the RedisAdapter logs info when connect is called while already connected.""" """Test that the RedisAdapter logs info when connect is called while already connected."""
with caplog.at_level(logging.INFO): with caplog.at_level(logging.INFO):
redis_adapter.connect() redis_adapter.connect()
assert "Already connected to Redis" in caplog.text assert "Already connected" in caplog.text
assert "Redis" in caplog.text
def test_should_raise_connection_error_when_unable_to_connect() -> None: def test_should_raise_connection_error_when_unable_to_connect() -> None:
+2 -2
View File
@@ -3,12 +3,12 @@
from __future__ import annotations from __future__ import annotations
import builtins import builtins
from collections.abc import Callable, Mapping, Sequence
import importlib import importlib
import os import os
from pathlib import Path
import subprocess import subprocess
import sys import sys
from collections.abc import Callable, Mapping, Sequence
from pathlib import Path
from types import ModuleType from types import ModuleType
from unittest.mock import patch from unittest.mock import patch
+1 -1
View File
@@ -4,9 +4,9 @@ from __future__ import annotations
from unittest.mock import MagicMock from unittest.mock import MagicMock
from minio import Minio
import pytest import pytest
import redis import redis
from minio import Minio
from python_repositories.adapters.minio_adapter import MinioAdapter from python_repositories.adapters.minio_adapter import MinioAdapter
from python_repositories.adapters.redis_adapter import RedisAdapter from python_repositories.adapters.redis_adapter import RedisAdapter
@@ -1,6 +1,7 @@
"""Unit tests for ConnectionAwareInterface.""" """Unit tests for ConnectionAwareInterface."""
import pytest import pytest
from python_repositories.interfaces.connection_aware_interface import ( from python_repositories.interfaces.connection_aware_interface import (
ConnectionAwareInterface, ConnectionAwareInterface,
) )
@@ -3,6 +3,7 @@
from __future__ import annotations from __future__ import annotations
import pytest import pytest
from python_repositories.interfaces.context_aware_interface import ContextAwareInterface from python_repositories.interfaces.context_aware_interface import ContextAwareInterface
@@ -3,6 +3,7 @@
from typing import Any from typing import Any
import pytest import pytest
from python_repositories.interfaces.json_repository_interface import ( from python_repositories.interfaces.json_repository_interface import (
JsonRepositoryInterface, JsonRepositoryInterface,
) )
+1 -1
View File
@@ -5,8 +5,8 @@ from __future__ import annotations
from dataclasses import replace from dataclasses import replace
from unittest.mock import MagicMock from unittest.mock import MagicMock
import pytest
from minio import Minio, S3Error from minio import Minio, S3Error
import pytest
from urllib3.response import BaseHTTPResponse from urllib3.response import BaseHTTPResponse
from python_repositories.adapters.minio_adapter import MinioAdapter from python_repositories.adapters.minio_adapter import MinioAdapter
@@ -3,6 +3,7 @@
from io import BytesIO from io import BytesIO
import pytest import pytest
from python_repositories.interfaces.object_repository_interface import ( from python_repositories.interfaces.object_repository_interface import (
ObjectRepositoryInterface, ObjectRepositoryInterface,
) )
Generated
+6 -6
View File
@@ -1,5 +1,5 @@
version = 1 version = 1
revision = 3 revision = 2
requires-python = ">=3.12" requires-python = ">=3.12"
resolution-markers = [ resolution-markers = [
"python_full_version >= '3.15'", "python_full_version >= '3.15'",
@@ -1056,7 +1056,7 @@ wheels = [
[[package]] [[package]]
name = "python-repositories" name = "python-repositories"
version = "2.0.2" version = "2.0.4"
source = { editable = "." } source = { editable = "." }
dependencies = [ dependencies = [
{ name = "python-dotenv" }, { name = "python-dotenv" },
@@ -1097,12 +1097,12 @@ provides-extras = ["redis", "minio"]
[package.metadata.requires-dev] [package.metadata.requires-dev]
dev = [ dev = [
{ name = "mypy", specifier = ">=1.17.1" }, { name = "mypy", specifier = ">=2.1.0" },
{ name = "pre-commit", specifier = ">=4.3.0" }, { name = "pre-commit", specifier = ">=4.6.0" },
{ name = "pytest", specifier = ">=8.4.1" }, { name = "pytest", specifier = ">=8.4.1" },
{ name = "pytest-cov", specifier = ">=7.0.0" }, { name = "pytest-cov", specifier = ">=7.0.0" },
{ name = "pyupgrade", specifier = ">=3.20.0" }, { name = "pyupgrade", specifier = ">=3.21.2" },
{ name = "ruff", specifier = ">=0.12.11" }, { name = "ruff", specifier = ">=0.15.20" },
{ name = "safety", specifier = ">=3.6.0" }, { name = "safety", specifier = ">=3.6.0" },
{ name = "testcontainers", specifier = ">=4.13.0" }, { name = "testcontainers", specifier = ">=4.13.0" },
{ name = "types-redis", specifier = ">=4.6.0.20241004" }, { name = "types-redis", specifier = ">=4.6.0.20241004" },