Compare commits

...
Author SHA1 Message Date
CI Bot 43c5e53c7f chore(deps): update dependencies [automated]
PR Title Check / check-title (pull_request) Successful in 5s
Test Python Package / unit-tests (pull_request) Successful in 23s
Code Quality Pipeline / code-quality (pull_request) Successful in 57s
Test Python Package / integration-tests (pull_request) Successful in 1m48s
Test Python Package / coverage-report (pull_request) Successful in 18s
2026-07-20 03:01:36 +00:00
brian ac82f71347 Merge pull request '[minor] Add QueueRepositoryInterface with memory and file-backed adapters' (#62) from cursor/queue-adapters into main
Release on merge to main / release (push) Failing after 15s
Test Python Package / unit-tests (push) Successful in 14s
Code Quality Pipeline / code-quality (push) Successful in 28s
Test Python Package / integration-tests (push) Successful in 23s
Test Python Package / coverage-report (push) Successful in 8s
Reviewed-on: #62
2026-07-16 21:27:49 +02:00
Brian Bjarke JensenandCursor 24d5ea14d6 Apply Prettier formatting to README and CHANGELOG.
PR Title Check / check-title (pull_request) Successful in 8s
Test Python Package / unit-tests (pull_request) Successful in 15s
Test Python Package / integration-tests (pull_request) Successful in 24s
Code Quality Pipeline / code-quality (pull_request) Successful in 31s
Test Python Package / coverage-report (pull_request) Successful in 12s
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-16 21:25:49 +02:00
Brian Bjarke JensenandCursor b8d8176877 Align changelog section headings with Prettier markdown spacing.
Test Python Package / unit-tests (pull_request) Successful in 13s
Test Python Package / coverage-report (pull_request) Successful in 13s
PR Title Check / check-title (pull_request) Successful in 7s
Code Quality Pipeline / code-quality (pull_request) Failing after 29s
Test Python Package / integration-tests (pull_request) Successful in 1m15s
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-16 21:24:06 +02:00
Brian Bjarke JensenandCursor a78320b434 Move queue helper functions to staticmethods on their adapters.
PR Title Check / check-title (pull_request) Successful in 8s
Test Python Package / unit-tests (pull_request) Successful in 20s
Code Quality Pipeline / code-quality (pull_request) Failing after 32s
Test Python Package / integration-tests (pull_request) Successful in 1m33s
Test Python Package / coverage-report (pull_request) Successful in 16s
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-16 21:07:35 +02:00
Brian Bjarke JensenandCursor 1822d886b1 Apply ruff formatting to queue adapter and tests.
PR Title Check / check-title (pull_request) Successful in 11s
Test Python Package / unit-tests (pull_request) Successful in 16s
Code Quality Pipeline / code-quality (pull_request) Failing after 30s
Test Python Package / integration-tests (pull_request) Successful in 34s
Test Python Package / coverage-report (pull_request) Successful in 16s
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-16 20:24:58 +02:00
Brian Bjarke JensenandCursor 4f58c32dd6 Add QueueRepositoryInterface with memory and file-backed adapters.
PR Title Check / check-title (pull_request) Successful in 9s
Code Quality Pipeline / code-quality (pull_request) Failing after 53s
Test Python Package / unit-tests (pull_request) Successful in 1m1s
Test Python Package / integration-tests (pull_request) Successful in 1m44s
Test Python Package / coverage-report (pull_request) Successful in 13s
Provide a generic disk-backed FIFO queue with configurable path, retention, and dedup keys so consumers can buffer items across restarts without optional extras.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-16 20:20:52 +02:00
brian 88ea7f06ab Merge pull request 'chore(deps): update dependencies' (#61) from renovate/auto-deps-update into main
Release on merge to main / release (push) Failing after 5s
Test Python Package / integration-tests (push) Failing after 1s
Test Python Package / unit-tests (push) Failing after 2s
Test Python Package / coverage-report (push) Has been skipped
Code Quality Pipeline / code-quality (push) Failing after 1m10s
Build CI Image / build-and-push (push) Successful in 1m40s
Reviewed-on: #61
2026-07-16 20:19:13 +02:00
CI Bot 2b5718ba41 chore(deps): update dependencies [automated]
PR Title Check / check-title (pull_request) Successful in 7s
Code Quality Pipeline / code-quality (pull_request) Successful in 31s
Test Python Package / unit-tests (pull_request) Successful in 15s
Test Python Package / integration-tests (pull_request) Successful in 41s
Test Python Package / coverage-report (pull_request) Successful in 13s
2026-07-13 03:00:38 +00:00
CI Bot b6538f9e91 chore: release v2.2.0 [skip ci] 2026-07-12 16:52:22 +00:00
brian b37b2e6e50 Merge pull request '[minor] Add Postgres table adapter with TableRepositoryInterface' (#58) from cursor/postgres-table-adapter into main
Release on merge to main / release (push) Failing after 9s
Code Quality Pipeline / code-quality (push) Successful in 20s
Test Python Package / unit-tests (push) Successful in 13s
Build CI Image / build-and-push (push) Successful in 38s
Test Python Package / integration-tests (push) Successful in 1m9s
Test Python Package / coverage-report (push) Successful in 13s
Reviewed-on: LilleVemmelund/python-repositories#58
2026-07-12 18:51:59 +02:00
Brian Bjarke JensenandCursor 1babb52d09 Apply ruff formatting to postgres adapter and tests.
PR Title Check / check-title (pull_request) Successful in 8s
Test Python Package / unit-tests (pull_request) Successful in 13s
Code Quality Pipeline / code-quality (pull_request) Successful in 24s
Test Python Package / integration-tests (pull_request) Successful in 29s
Test Python Package / coverage-report (pull_request) Successful in 10s
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-11 15:01:58 +02:00
Brian Bjarke JensenandCursor 63ee17d544 Apply prettier formatting to README and CHANGELOG.
PR Title Check / check-title (pull_request) Successful in 8s
Test Python Package / unit-tests (pull_request) Successful in 10s
Code Quality Pipeline / code-quality (pull_request) Failing after 18s
Test Python Package / integration-tests (pull_request) Successful in 56s
Test Python Package / coverage-report (pull_request) Successful in 13s
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-11 15:01:08 +02:00
Brian Bjarke JensenandCursor 2e38bd2406 Fix ruff import ordering in postgres test files.
PR Title Check / check-title (pull_request) Successful in 8s
Test Python Package / unit-tests (pull_request) Successful in 10s
Code Quality Pipeline / code-quality (pull_request) Failing after 17s
Test Python Package / integration-tests (pull_request) Successful in 47s
Test Python Package / coverage-report (pull_request) Successful in 12s
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-11 15:00:01 +02:00
Brian Bjarke JensenandCursor dc6f8a3e89 Add Postgres table adapter with TableRepositoryInterface.
PR Title Check / check-title (pull_request) Successful in 7s
Code Quality Pipeline / code-quality (pull_request) Failing after 19s
Test Python Package / unit-tests (pull_request) Successful in 47s
Test Python Package / integration-tests (pull_request) Successful in 44s
Test Python Package / coverage-report (pull_request) Successful in 11s
Introduce PostgresAdapter for dict-based row CRUD via psycopg3, including config, lazy exports, unit/integration tests with testcontainers, and an example UserTableRepository.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-11 14:57:34 +02:00
CI Bot f28d3c4844 chore: release v2.1.0 [skip ci] 2026-07-11 08:56:19 +00:00
brian 1cbcc1c19b Merge pull request '[minor] Add runtime-checkable Protocol typing to public interfaces' (#57) from cursor/protocol-abc-interfaces into main
Test Python Package / unit-tests (push) Successful in 10s
Release on merge to main / release (push) Successful in 15s
Code Quality Pipeline / code-quality (push) Successful in 23s
Test Python Package / integration-tests (push) Successful in 20s
Test Python Package / coverage-report (push) Successful in 7s
Reviewed-on: LilleVemmelund/python-repositories#57
2026-07-11 10:56:01 +02:00
Brian Bjarke JensenandCursor a0e5c9dcca Document test organization and add Cursor workflow rules.
PR Title Check / check-title (pull_request) Successful in 8s
Test Python Package / unit-tests (pull_request) Successful in 13s
Code Quality Pipeline / code-quality (pull_request) Successful in 23s
Test Python Package / integration-tests (pull_request) Successful in 1m14s
Test Python Package / coverage-report (pull_request) Successful in 12s
Record the one-to-one unit test file convention in README and add rules for test placement and copy-ready PR text after first push.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-11 10:50:51 +02:00
Brian Bjarke JensenandCursor 44b15cb21a Move structural typing tests into interface unit test files.
Code Quality Pipeline / code-quality (pull_request) Successful in 24s
PR Title Check / check-title (pull_request) Successful in 35s
Test Python Package / coverage-report (pull_request) Successful in 11s
Test Python Package / unit-tests (pull_request) Successful in 12s
Test Python Package / integration-tests (pull_request) Successful in 26s
Align test layout with the one-test-file-per-interface convention by removing structural_typing_test.py.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-11 10:32:17 +02:00
Brian Bjarke JensenandCursor f6ee9a3724 Add runtime-checkable Protocol typing to all public interfaces.
PR Title Check / check-title (pull_request) Successful in 7s
Test Python Package / unit-tests (pull_request) Successful in 9s
Code Quality Pipeline / code-quality (pull_request) Successful in 22s
Test Python Package / integration-tests (pull_request) Successful in 24s
Test Python Package / coverage-report (pull_request) Successful in 8s
Enables structural subtyping for consumers while preserving nominal adapter inheritance, instantiation guards, and scan_keys defaults.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-11 09:45:42 +02:00
brian 0829354cc4 Merge pull request 'Add async future note and remove redundant pytest pythonpath' (#56) from cursor/readme-async-and-test-path-cleanup into main
Release on merge to main / release (push) Successful in 7s
Test Python Package / unit-tests (push) Successful in 12s
Code Quality Pipeline / code-quality (push) Successful in 23s
Test Python Package / integration-tests (push) Successful in 24s
Build CI Image / build-and-push (push) Successful in 49s
Test Python Package / coverage-report (push) Successful in 9s
Reviewed-on: LilleVemmelund/python-repositories#56
2026-07-11 09:28:01 +02:00
Brian Bjarke JensenandCursor a0d509a2a3 Format CHANGELOG.md with Prettier.
PR Title Check / check-title (pull_request) Successful in 7s
Test Python Package / unit-tests (pull_request) Successful in 12s
Code Quality Pipeline / code-quality (pull_request) Successful in 23s
Test Python Package / integration-tests (pull_request) Successful in 1m14s
Test Python Package / coverage-report (pull_request) Successful in 13s
Fixes pre-commit prettier check in CI.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-11 09:25:16 +02:00
Brian Bjarke JensenandCursor 9b7b15a67e Document possible async support and drop redundant pytest pythonpath.
Test Python Package / unit-tests (pull_request) Successful in 12s
Code Quality Pipeline / code-quality (pull_request) Failing after 23s
PR Title Check / check-title (pull_request) Successful in 32s
Test Python Package / integration-tests (pull_request) Successful in 25s
Test Python Package / coverage-report (pull_request) Successful in 11s
The package is already installed via uv sync, so pytest no longer needs a manual repo-root path.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-11 09:22:37 +02:00
CI Bot cd5b84b88e chore: release v2.0.5 [skip ci] 2026-07-10 19:44:15 +00:00
brian 6d02f32ca5 Merge pull request '[patch] Allow empty payloads in Redis and MinIO adapters' (#55) from cursor/allow-empty-payloads into main
Release on merge to main / release (push) Successful in 15s
Test Python Package / integration-tests (push) Successful in 23s
Test Python Package / coverage-report (push) Successful in 10s
Test Python Package / unit-tests (push) Successful in 13s
Code Quality Pipeline / code-quality (push) Successful in 19s
Reviewed-on: LilleVemmelund/python-repositories#55
2026-07-10 21:43:59 +02:00
Brian Bjarke JensenandCursor 57b396bcd3 Allow empty payloads in Redis and MinIO adapters.
PR Title Check / check-title (pull_request) Successful in 8s
Test Python Package / unit-tests (pull_request) Successful in 12s
Code Quality Pipeline / code-quality (pull_request) Successful in 20s
Test Python Package / integration-tests (pull_request) Successful in 24s
Test Python Package / coverage-report (pull_request) Successful in 11s
Relax write validation so {} and zero-byte BytesIO round-trip correctly,
document None-vs-empty semantics on interfaces and adapters, and add
integration tests for placeholders and existence distinction.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-10 21:39:51 +02:00
brian 093e538d5b Merge pull request '[chore] Scope integration fixtures per backend subdirectory' (#54) from cursor/scoped-integration-fixtures into main
Release on merge to main / release (push) Successful in 7s
Code Quality Pipeline / code-quality (push) Successful in 19s
Test Python Package / unit-tests (push) Successful in 14s
Build CI Image / build-and-push (push) Successful in 38s
Test Python Package / integration-tests (push) Successful in 50s
Test Python Package / coverage-report (push) Successful in 12s
Reviewed-on: LilleVemmelund/python-repositories#54
2026-07-10 21:16:27 +02:00
Brian Bjarke JensenandCursor b28ac6e803 Scope integration fixtures per backend subdirectory.
PR Title Check / check-title (pull_request) Successful in 7s
Test Python Package / unit-tests (pull_request) Successful in 13s
Test Python Package / integration-tests (pull_request) Successful in 23s
Test Python Package / coverage-report (pull_request) Successful in 10s
Code Quality Pipeline / code-quality (pull_request) Successful in 49s
Split Redis and MinIO container setup into backend-specific conftest modules so only the containers needed for collected tests are imported and started.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-10 21:11:53 +02:00
brian 6c66fe4e21 Merge pull request 'Add consumer-facing CHANGELOG.md maintained by release CI' (#53) from cursor/consumer-changelog into main
Release on merge to main / release (push) Successful in 7s
Code Quality Pipeline / code-quality (push) Successful in 19s
Test Python Package / unit-tests (push) Successful in 13s
Build CI Image / build-and-push (push) Successful in 36s
Test Python Package / integration-tests (push) Successful in 20s
Test Python Package / coverage-report (push) Successful in 11s
Reviewed-on: LilleVemmelund/python-repositories#53
2026-07-10 20:40:34 +02:00
Brian Bjarke Jensen 089a1287f8 Format CHANGELOG.md with Prettier
PR Title Check / check-title (pull_request) Successful in 6s
Test Python Package / unit-tests (pull_request) Successful in 9s
Code Quality Pipeline / code-quality (pull_request) Successful in 23s
Test Python Package / integration-tests (pull_request) Successful in 24s
Test Python Package / coverage-report (pull_request) Successful in 10s
2026-07-10 20:38:40 +02:00
Brian Bjarke JensenandCursor 0a5bd39801 Add consumer-facing CHANGELOG.md maintained by release CI.
PR Title Check / check-title (pull_request) Successful in 8s
Test Python Package / unit-tests (pull_request) Successful in 12s
Code Quality Pipeline / code-quality (pull_request) Failing after 23s
Test Python Package / integration-tests (pull_request) Successful in 45s
Test Python Package / coverage-report (pull_request) Successful in 11s
Backfill version history from git tags and update CHANGELOG automatically on each release alongside Gitea release notes.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-10 20:28:46 +02:00
brian 683ee332d3 Merge pull request '[chore] add repository URL and Python 3.12 packaging metadata' (#52) from cursor/add-packaging-metadata into main
Release on merge to main / release (push) Successful in 7s
Test Python Package / unit-tests (push) Successful in 14s
Code Quality Pipeline / code-quality (push) Successful in 22s
Test Python Package / integration-tests (push) Successful in 27s
Test Python Package / coverage-report (push) Successful in 13s
Build CI Image / build-and-push (push) Successful in 1m0s
Reviewed-on: LilleVemmelund/python-repositories#52
2026-07-10 19:50:22 +02:00
Brian Bjarke JensenandCursor 9df5032d44 Add repository URL and Python 3.12 packaging metadata.
Test Python Package / coverage-report (pull_request) Successful in 10s
PR Title Check / check-title (pull_request) Successful in 8s
Test Python Package / unit-tests (pull_request) Successful in 12s
Test Python Package / integration-tests (pull_request) Successful in 24s
Code Quality Pipeline / code-quality (pull_request) Successful in 51s
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-10 19:46:21 +02:00
brian 4f6f33f36e Merge pull request 'Drop types-redis in favor of redis inline types' (#51) from cursor/drop-types-redis into main
Release on merge to main / release (push) Successful in 5s
Test Python Package / unit-tests (push) Successful in 12s
Code Quality Pipeline / code-quality (push) Successful in 23s
Build CI Image / build-and-push (push) Successful in 1m3s
Test Python Package / integration-tests (push) Successful in 57s
Test Python Package / coverage-report (push) Successful in 10s
Reviewed-on: LilleVemmelund/python-repositories#51
2026-07-10 19:28:20 +02:00
Brian Bjarke JensenandCursor 10d43ce198 Drop types-redis in favor of redis inline types.
PR Title Check / check-title (pull_request) Successful in 7s
Test Python Package / unit-tests (pull_request) Successful in 12s
Test Python Package / integration-tests (pull_request) Successful in 25s
Test Python Package / coverage-report (pull_request) Successful in 11s
Code Quality Pipeline / code-quality (pull_request) Successful in 47s
The outdated redis 4.6 stubs conflicted with redis 6+ py.typed annotations used at runtime.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-10 19:24:17 +02:00
CI Bot 4f7b67203a chore: release v2.0.4 [skip ci] 2026-07-10 12:47:15 +00:00
brian a27e9e5763 Merge pull request '[fix] Include optional dependencies in security audit' (#50) from cursor/security-audit-all-extras into main
Test Python Package / unit-tests (push) Successful in 10s
Release on merge to main / release (push) Successful in 14s
Code Quality Pipeline / code-quality (push) Successful in 21s
Test Python Package / integration-tests (push) Successful in 20s
Test Python Package / coverage-report (push) Successful in 8s
Reviewed-on: LilleVemmelund/python-repositories#50
2026-07-10 14:47:01 +02:00
Brian Bjarke JensenandCursor fd0045d3e0 Include optional dependencies in security audit.
PR Title Check / check-title (pull_request) Successful in 5s
Test Python Package / unit-tests (pull_request) Successful in 12s
Code Quality Pipeline / code-quality (pull_request) Successful in 21s
Test Python Package / integration-tests (pull_request) Successful in 19s
Test Python Package / coverage-report (pull_request) Successful in 8s
Sync all extras before safety check so redis and minio vulnerabilities are scanned.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-10 14:44:35 +02:00
brian 515f4bbeae Merge pull request 'revert 707f11f578f007c7152582ea524b66dacea50111' (#49) from brian-patch-1 into main
Release on merge to main / release (push) Successful in 5s
Test Python Package / unit-tests (push) Successful in 10s
Code Quality Pipeline / code-quality (push) Successful in 20s
Test Python Package / integration-tests (push) Successful in 19s
Test Python Package / coverage-report (push) Successful in 7s
Build CI Image / build-and-push (push) Successful in 1m32s
Reviewed-on: LilleVemmelund/python-repositories#49
2026-07-10 14:42:24 +02:00
brian ef31fb4b4e revert 707f11f578
revert chore: release v2.0.4 [skip ci]
2026-07-10 14:42:09 +02:00
CI Bot 707f11f578 chore: release v2.0.4 [skip ci] 2026-07-10 12:36:58 +00:00
brian b1d34f8f0f Merge pull request '[patch] Unify dev tooling via uv so pre-commit, CI, and the update bot stay aligned' (#48) from chore/align-dev-tooling-via-uv into main
Test Python Package / unit-tests (push) Successful in 13s
Code Quality Pipeline / code-quality (push) Successful in 21s
Test Python Package / integration-tests (push) Successful in 20s
Test Python Package / coverage-report (push) Successful in 8s
Build CI Image / build-and-push (push) Successful in 58s
Release on merge to main / release (push) Failing after 9s
Reviewed-on: LilleVemmelund/python-repositories#48
2026-07-10 14:36:44 +02:00
Brian Bjarke JensenandCursor 565879dd52 Unify dev tooling via uv so pre-commit, CI, and the update bot stay aligned.
Test Python Package / unit-tests (pull_request) Successful in 12s
Code Quality Pipeline / code-quality (pull_request) Successful in 19s
PR Title Check / check-title (pull_request) Successful in 31s
Test Python Package / integration-tests (pull_request) Successful in 23s
Test Python Package / coverage-report (pull_request) Successful in 10s
Route Python hooks through uv run with versions pinned in uv.lock, add explicit
Ruff settings, and extend the dependency bot to run pre-commit autoupdate.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-10 14:29:22 +02:00
brian 0d0f430f6f Merge pull request 'Fix automated package publishing in release workflow' (#47) from fix/release-workflow-publish into main
Release on merge to main / release (push) Successful in 5s
Test Python Package / unit-tests (push) Successful in 12s
Code Quality Pipeline / code-quality (push) Successful in 23s
Test Python Package / integration-tests (push) Successful in 19s
Test Python Package / coverage-report (push) Successful in 8s
Reviewed-on: LilleVemmelund/python-repositories#47
2026-07-10 14:17:10 +02:00
Brian Bjarke JensenandCursor 5ac40b1ba5 Publish packages from the release workflow so automated releases reach the registry.
Release commits use [skip ci], which prevents publish.yml from running on tag push.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-10 14:15:04 +02:00
65 changed files with 4459 additions and 686 deletions
+58
View File
@@ -0,0 +1,58 @@
---
description: After first branch push, provide copy-ready PR title and body
alwaysApply: true
---
# PR Title and Comment After First Push
The user creates pull requests manually. After the **first push** of a feature branch to remote in a session, always end your response with a copy-ready PR title and PR comment.
## When to provide it
- After the first successful `git push` (or `git push -u origin <branch>`) for a branch in the current task
- Do **not** create the PR with `gh pr create` unless explicitly asked
- On later pushes to the same branch, only repeat if the user asks or the change set meaningfully updates what the PR should say
## PR title
- Use a fitting version-bump prefix when changes touch `python_repositories/`:
- `[patch]` or `[fix]` — bug fix
- `[minor]` or `[feat]` — new feature
- `[major]` or `[breaking]` — breaking change
- Docs-, CI-, or test-only changes: no prefix required
- Keep the title concise and descriptive
## PR comment format
Provide two separate fenced code blocks the user can copy directly:
1. **PR title** — single line in a code block
2. **PR comment** — body with `## Summary` and `## Test plan` sections, checklist items reflecting what was actually run
Example structure:
````
PR title:
```
[minor] Short description of the change
```
PR comment:
```
## Summary
- Bullet points of what changed and why
## Test plan
- [x] Commands or checks that were run
```
````
## Notes
- Match the repo PR template ([`.gitea/PULL_REQUEST_TEMPLATE.md`](.gitea/PULL_REQUEST_TEMPLATE.md))
- Be accurate: only mark test-plan items done if they were run
- Keep prose clear; the user copies this verbatim into Gitea
+31
View File
@@ -0,0 +1,31 @@
---
description: Unit test file naming and placement conventions
globs: tests/**/*.py
alwaysApply: false
---
# Test Organization
Unit tests use a one-to-one mapping between source modules and test files.
## Naming
- `python_repositories/<path>/<module>.py` → `tests/unit/<module>_test.py`
- Test files must end with `_test.py` (enforced by pre-commit `name-tests-test`)
## Placement
- Add tests to the existing `*_test.py` for the module under test
- Do not create cross-cutting test files (e.g. `structural_typing_test.py`); colocate with the relevant interface/adapter/config test file
- Shared fixtures → `tests/conftest.py`
- Module-specific helper classes → the matching test file
## Examples
```
json_repository_interface.py → tests/unit/json_repository_interface_test.py
redis_adapter.py → tests/unit/redis_adapter_test.py
redis_config.py → tests/unit/redis_config_test.py
```
Integration tests live under `tests/integration/<backend>/` with the same `_test.py` suffix.
+9
View File
@@ -8,3 +8,12 @@ MINIO_SECRET_KEY=minioadmin
MINIO_BUCKET=my-bucket
MINIO_SECURE=false
MINIO_CREATE_BUCKET_IF_MISSING=true
# Postgres (requires postgres extra)
POSTGRES_URI=postgresql://localhost/mydb
POSTGRES_TABLE=users
POSTGRES_PRIMARY_KEY=id
# File-backed queue (no optional extra)
FILE_QUEUE_PATH=/tmp/python-repositories-queue.jsonl
# FILE_QUEUE_MAX_AGE_HOURS=24
@@ -14,12 +14,38 @@ jobs:
with:
token: ${{ secrets.CI_RUNNER_TOKEN }}
- name: Sync dependencies
env:
UV_LINK_MODE: copy
UV_INDEX_GITEA_USERNAME: ci-bot
UV_INDEX_GITEA_PASSWORD: ${{ secrets.CI_RUNNER_TOKEN }}
run: uv sync --all-extras
- name: Upgrade dependencies
env:
UV_INDEX_GITEA_USERNAME: ci-bot
UV_INDEX_GITEA_PASSWORD: ${{ secrets.CI_RUNNER_TOKEN }}
run: uv lock --upgrade
- name: Update pre-commit hook revs
env:
UV_INDEX_GITEA_USERNAME: ci-bot
UV_INDEX_GITEA_PASSWORD: ${{ secrets.CI_RUNNER_TOKEN }}
run: uv run pre-commit autoupdate
- name: Sync upgraded dependencies
env:
UV_LINK_MODE: copy
UV_INDEX_GITEA_USERNAME: ci-bot
UV_INDEX_GITEA_PASSWORD: ${{ secrets.CI_RUNNER_TOKEN }}
run: uv sync --all-extras
- name: Smoke check
run: |
uv run ruff check .
uv run ruff format --check .
uv run mypy .
- name: Commit and push changes
env:
API_URL: ${{ vars.API_URL }}
+30 -1
View File
@@ -40,6 +40,15 @@ jobs:
NEW_TAG="v${{ steps.bump.outputs.version }}"
scripts/ci/generate-release-notes.sh "$NEW_TAG" "$PR_TITLE" "$PREV_TAG"
- name: Update CHANGELOG
if: steps.meta.outputs.bump != 'skip'
env:
PR_TITLE: ${{ steps.meta.outputs.pr_title }}
run: |
PREV_TAG=$(git describe --tags --abbrev=0)
VERSION="${{ steps.bump.outputs.version }}"
scripts/ci/update-changelog.sh "$VERSION" "$PR_TITLE" "$PREV_TAG"
- name: Commit, tag, and push
if: steps.meta.outputs.bump != 'skip'
env:
@@ -48,7 +57,7 @@ jobs:
run: |
git config user.name "CI Bot"
git config user.email "ci-bot@example.com"
git add pyproject.toml uv.lock
git add pyproject.toml uv.lock CHANGELOG.md
git commit -m "chore: release v${VERSION} [skip ci]"
git tag "v${VERSION}"
git remote set-url origin "https://x-access-token:${CI_RUNNER_TOKEN}@${GITHUB_SERVER_URL#https://}/${GITHUB_REPOSITORY}.git"
@@ -74,3 +83,23 @@ jobs:
-H "Authorization: token ${CI_RUNNER_TOKEN}" \
-H "Content-Type: application/json" \
-d "$PAYLOAD"
# Publish here instead of relying on publish.yml tag trigger: release commits use
# [skip ci], which suppresses tag-push workflows, and Gitea does not emit release
# events for releases created by the CI bot account.
- name: Build package
if: steps.meta.outputs.bump != 'skip'
env:
UV_LINK_MODE: copy
UV_INDEX_GITEA_USERNAME: ci-bot
UV_INDEX_GITEA_PASSWORD: ${{ secrets.CI_RUNNER_TOKEN }}
run: |
uv sync --no-dev --frozen
uv build
- name: Publish to Gitea Package Registry
if: steps.meta.outputs.bump != 'skip'
env:
UV_PUBLISH_TOKEN: ${{ secrets.CI_RUNNER_TOKEN }}
UV_PUBLISH_URL: ${{ vars.REPOSITORY_URL }}
run: uv publish
+1 -1
View File
@@ -17,7 +17,7 @@ jobs:
UV_LINK_MODE: copy
UV_INDEX_GITEA_USERNAME: ci-bot
UV_INDEX_GITEA_PASSWORD: ${{ secrets.CI_RUNNER_TOKEN }}
run: uv sync --frozen
run: uv sync --all-extras --frozen
- name: Run safety check
run: uv run safety check
+21 -19
View File
@@ -1,7 +1,7 @@
repos:
# General repository hygiene hooks
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.5.0
rev: v6.0.0
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
@@ -11,32 +11,34 @@ repos:
- id: name-tests-test
- id: check-merge-conflict
# Python linting and formatting with Ruff
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.12.8
# Python tooling via uv (versions pinned in uv.lock)
- repo: local
hooks:
- id: ruff-check
args: [--fix]
name: ruff check
entry: uv run ruff check --fix
language: system
types: [python]
- id: ruff-format
# Static type checking with mypy
- repo: https://github.com/pre-commit/mirrors-mypy
rev: v1.8.0
hooks:
name: ruff format
entry: uv run ruff format
language: system
types: [python]
- id: mypy
additional_dependencies:
- types-redis
# Python syntax modernization with pyupgrade
- repo: https://github.com/asottile/pyupgrade
rev: v3.20.0
hooks:
name: mypy
entry: uv run mypy .
language: system
types: [python]
pass_filenames: false
- id: pyupgrade
args: ["--py312-plus"]
name: pyupgrade
entry: uv run pyupgrade --py312-plus
language: system
types: [python]
# Formatting for Markdown, JSON, and YAML with Prettier
- repo: https://github.com/pre-commit/mirrors-prettier
rev: v3.1.0
rev: v4.0.0-alpha.8
hooks:
- id: prettier
files: "\\.(md|json|yaml|yml)$"
+375
View File
@@ -0,0 +1,375 @@
# Changelog
All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased]
## [2.2.0] - 2026-07-12
### Summary
Add Postgres table adapter with TableRepositoryInterface
### Changed
- b37b2e6 Merge pull request '[minor] Add Postgres table adapter with TableRepositoryInterface' (#58) from cursor/postgres-table-adapter into main
- 1babb52 Apply ruff formatting to postgres adapter and tests.
- 63ee17d Apply prettier formatting to README and CHANGELOG.
- 2e38bd2 Fix ruff import ordering in postgres test files.
- dc6f8a3 Add Postgres table adapter with TableRepositoryInterface.
## [2.1.0] - 2026-07-11
### Summary
Add runtime-checkable Protocol typing to public interfaces
### Changed
- 1cbcc1c Merge pull request '[minor] Add runtime-checkable Protocol typing to public interfaces' (#57) from cursor/protocol-abc-interfaces into main
- a0e5c9d Document test organization and add Cursor workflow rules.
- 44b15cb Move structural typing tests into interface unit test files.
- f6ee9a3 Add runtime-checkable Protocol typing to all public interfaces.
- 0829354 Merge pull request 'Add async future note and remove redundant pytest pythonpath' (#56) from cursor/readme-async-and-test-path-cleanup into main
- a0d509a Format CHANGELOG.md with Prettier.
- 9b7b15a Document possible async support and drop redundant pytest pythonpath.
## [2.0.5] - 2026-07-10
### Summary
Allow empty payloads in Redis and MinIO adapters
### Changed
- 6d02f32 Merge pull request '[patch] Allow empty payloads in Redis and MinIO adapters' (#55) from cursor/allow-empty-payloads into main
- 57b396b Allow empty payloads in Redis and MinIO adapters.
- 093e538 Merge pull request '[chore] Scope integration fixtures per backend subdirectory' (#54) from cursor/scoped-integration-fixtures into main
- b28ac6e Scope integration fixtures per backend subdirectory.
- 6c66fe4 Merge pull request 'Add consumer-facing CHANGELOG.md maintained by release CI' (#53) from cursor/consumer-changelog into main
- 089a128 Format CHANGELOG.md with Prettier
- 0a5bd39 Add consumer-facing CHANGELOG.md maintained by release CI.
- 683ee33 Merge pull request '[chore] add repository URL and Python 3.12 packaging metadata' (#52) from cursor/add-packaging-metadata into main
- 9df5032 Add repository URL and Python 3.12 packaging metadata.
- 4f6f33f Merge pull request 'Drop types-redis in favor of redis inline types' (#51) from cursor/drop-types-redis into main
- 10d43ce Drop types-redis in favor of redis inline types.
## [2.0.4] - 2026-07-10
### Summary
Include optional dependencies in security audit
### Changed
- 4f7b672 chore: release v2.0.4 [skip ci]
- a27e9e5 Merge pull request '[fix] Include optional dependencies in security audit' (#50) from cursor/security-audit-all-extras into main
- fd0045d Include optional dependencies in security audit.
- 515f4bb Merge pull request 'revert 707f11f578f007c7152582ea524b66dacea50111' (#49) from brian-patch-1 into main
- ef31fb4 revert 707f11f578f007c7152582ea524b66dacea50111
- 707f11f chore: release v2.0.4 [skip ci]
- b1d34f8 Merge pull request '[patch] Unify dev tooling via uv so pre-commit, CI, and the update bot stay aligned' (#48) from chore/align-dev-tooling-via-uv into main
- 565879d Unify dev tooling via uv so pre-commit, CI, and the update bot stay aligned.
- 0d0f430 Merge pull request 'Fix automated package publishing in release workflow' (#47) from fix/release-workflow-publish into main
- 5ac40b1 Publish packages from the release workflow so automated releases reach the registry.
## [2.0.3] - 2026-07-10
### Summary
Use structured fields for adapter debug and info logs
### Changed
- fa4ff1f chore: release v2.0.3 [skip ci]
- 0a39151 Merge pull request '[patch] Use structured fields for adapter debug and info logs' (#46) from cursor/structured-adapter-logging into main
- 82a702d Fix mypy error in Minio adapter log assertion.
- b886a7c Use structured fields for adapter debug and info logs.
- 4518a93 Merge pull request 'Migrate Python workflows to python-repositories-ci runner' (#41) from cursor/migrate-python-repositories-ci into main
- a8b7b59 Migrate Python workflows to python-repositories-ci runner label.
- 0e6941d Merge pull request 'Use lowercase CI image registry owner for Docker compatibility' (#45) from cursor/fix-ci-registry-owner-lowercase into main
- a6428cc Use lowercase CI image registry owner for Docker compatibility.
- 655fdc4 Merge pull request 'Fix CI image registry owner to match Gitea org slug' (#44) from cursor/fix-ci-registry-owner-case into main
- e031c54 Fix CI image registry owner to match Gitea org slug.
- 49b6238 Merge pull request 'Rename CI registry owner from brian to lille-vemmelund' (#43) from cursor/rename-ci-owner-to-lille-vemmelund into main
- 0549358 Keep python-utils package index under brian owner.
- 33efbd1 Update CI registry owner from brian to lille-vemmelund.
- 24126e4 Merge pull request 'Fix CI image build failures on runners with isolated DinD DNS' (#42) from fix/ci-image-build-dns into main
- f3b6cdb Fix CI image build failures on runners with isolated DinD DNS.
- fa2554e Merge pull request 'Add nightly CI base image and migrate workflows to python-repositories-ci' (#40) from cursor/ci-base-image into main
- b21fd24 Revert workflows to ubuntu-latest until CI image is seeded.
- 441a212 Add local CI image build script and document bootstrap flow.
- f2a8581 Read Python version from .python-version in CI image build.
- 3b05260 Add nightly CI base image and migrate workflows to python-repositories-ci.
## [2.0.2] - 2026-07-09
### Summary
Use dict[str, Any] for JSON repository document types
### Changed
- 369295b chore: release v2.0.2 [skip ci]
- 95c7859 Merge pull request '[patch] Use dict[str, Any] for JSON repository document types' (#38) from cursor/json-dict-str-any-typing into main
- 42e885e Use dict[str, Any] for JSON repository document types.
## [2.0.1] - 2026-07-09
### Summary
Align Redis and MinIO connect() idempotency
### Changed
- bd4794d chore: release v2.0.1 [skip ci]
- 3dc412c Merge pull request '[patch] Align Redis and MinIO connect() idempotency' (#37) from cursor/connect-idempotency-alignment into main
- b8d88e6 Cover injected-client connect when client was cleared by disconnect.
- 67bb88f Align Redis and MinIO connect() idempotency in ConnectionAwareAdapter.
## [2.0.0] - 2026-07-09
### Summary
Make MinIO bucket creation opt-in with production-safe defaults
### Changed
- c2e6a96 chore: release v2.0.0 [skip ci]
- 547a1b3 Merge pull request '[breaking] Make MinIO bucket creation opt-in with production-safe defaults' (#36) from cursor/minio-bucket-creation-opt-in into main
- 39baf9b Fix examples integration test by provisioning MinIO bucket in fixture.
- 4a5b3b6 Make MinIO bucket creation opt-in with production-safe defaults.
- 1431f45 Merge pull request 'Move Redis test container helpers into integration conftest' (#35) from cursor/move-redis-container-to-conftest into main
- eafc717 Move Redis test container helpers into integration conftest.
## [1.1.0] - 2026-07-08
### Summary
Add Redis scan_keys iterator API
### Changed
- 9a9b7b2 chore: release v1.1.0 [skip ci]
- 5b9573d Merge pull request '[minor] Add Redis scan_keys iterator API' (#34) from cursor/redis-scan-keys into main
- 50444af Add Redis scan_keys iterator API.
## [1.0.0] - 2026-07-08
### Summary
Clarify MinIO get() error semantics to match Redis behavior
### Changed
- 2f19fcc chore: release v1.0.0 [skip ci]
- 3bd6589 Merge pull request '[breaking] Clarify MinIO get() error semantics to match Redis behavior' (#33) from cursor/minio-error-semantics into main
- 5311d49 Clarify MinIO get() error semantics to match Redis behavior.
## [0.5.1] - 2026-07-07
### Summary
Fix MinIO get_object response leak in MinioAdapter.get()
### Changed
- 703bb95 chore: release v0.5.1 [skip ci]
- 8e47ebf Merge pull request '[patch] Fix MinIO get_object response leak in MinioAdapter.get()' (#32) from fix/minio-get-object-response-cleanup into main
- 9a800bf Fetch live PR title in pr-title-check CI workflow.
- 8c34534 Fix MinIO get_object response leak in MinioAdapter.get().
- b1210bd Merge pull request 'Reach 100% combined coverage with targeted unit tests' (#31) from cursor/balanced-100-coverage into main
- 39b1043 Fix mypy unreachable-code warning in minio reconnect test.
- beb2b51 Reach 100% combined coverage with targeted unit tests.
- ce062be Merge pull request 'Enforce 90% combined coverage floor in CI' (#29) from cursor/enforce-coverage-floor into main
- a381d45 Lower combined coverage floor from 95% to 90%.
- 5c8cd84 Document where combined coverage floor is enforced in CI.
- 3463298 Skip coverage floor check in partial CI test jobs.
- 5393efc Enforce a 95% combined coverage floor in CI and local runs.
- e39fc96 Merge pull request 'Split CI into parallel unit, integration, and coverage report jobs' (#28) from cursor/split-ci-test-jobs into main
- c1f4826 Use christopherhx v4 artifact actions for parallel coverage merge.
- 0315a33 Avoid cross-job artifacts; combine coverage with --cov-append.
- 5d33ec6 Use artifact action v3 for Gitea Actions compatibility.
- 9538e4d Split CI into parallel unit, integration, and coverage report jobs.
- 14b8d27 Merge pull request 'Replace hardcoded version examples in release docs' (#27) from cursor/fix-version-doc-drift into main
- d3cbb65 Use version-agnostic examples in release docs.
## [0.5.0] - 2026-07-06
### Summary
Add config and client injection with test reorganization
### Changed
- 7dbb1fc chore: release v0.5.0 [skip ci]
- 918e5e6 Merge pull request '[minor] Add config and client injection with test reorganization' (#25) from cursor/config-client-injection into main
- 3aa9128 Load MINIO_SECURE from env and replace adapter asserts with explicit errors.
- 7991dab Add config and client injection with test reorganization.
- 366831a Merge pull request 'chore(deps): update dependencies' (#26) from renovate/auto-deps-update into main
- a98905d chore(deps): update dependencies [automated]
- 7ed1b34 Merge pull request 'Sync uv.lock on release to prevent CI failures' (#24) from cursor/sync-uv-lock-on-release into main
- 3236f27 manually updated package version
- ac487d9 Sync uv.lock on release to keep CI clean.
## [0.4.1] - 2026-07-05
### Summary
Extract shared connection lifecycle into ConnectionAwareAdapter
### Changed
- a049ce3 chore: release v0.4.1 [skip ci]
- 7b4cf84 Merge pull request '[patch] Extract shared connection lifecycle into ConnectionAwareAdapter' (#23) from cursor/extract-connection-aware-adapter into main
- b5a92e6 Sync uv.lock with pyproject.toml version 0.4.0.
- 7b950e0 Extract shared connection lifecycle into ConnectionAwareAdapter.
## [0.4.0] - 2026-07-05
### Summary
Strengthen is_connected with cached health probes
### Changed
- fe0c477 chore: release v0.4.0 [skip ci]
- e7e0fc8 Merge pull request '[minor] Strengthen is_connected with cached health probes' (#22) from cursor/strengthen-is-connected-health-checks into main
- 34fc046 Sync uv.lock with pyproject.toml version 0.3.2.
- 5149299 Strengthen is_connected with cached health probes.
## [0.3.2] - 2026-06-30
### Summary
Fix optional dependency handling for Redis and MinIO adapters
### Changed
- fef9552 chore: release v0.3.2 [skip ci]
- e9b2189 Merge pull request '[patch] Fix optional dependency handling for Redis and MinIO adapters' (#19) from cursor/fix-optional-dependencies into main
- f595390 Fix optional-deps import test for full-suite execution.
- 1f02195 Delegate top-level adapter imports to adapters package.
- f092ca2 Fix trailing whitespace in LICENSE and .gitignore.
- c6eed43 Fix optional dependency handling for Redis and MinIO adapters.
- 9e4b1e2 Merge pull request 'Avoid testcontainers Redis deprecation warning in integration tests.' (#18) from fix/redis-testcontainer-deprecation into main
- 0096df7 Avoid testcontainers Redis deprecation warning in integration tests.
- 3073bfb Merge pull request 'Fix PR coverage comment posting in CI.' (#17) from fix/coverage-pr-comment into main
- 30a1de3 Fix PR coverage comment posting in CI.
- 8465957 Merge pull request '[patch] update dependencies' (#16) from renovate/auto-deps-update into main
- 3ecb9c3 chore: retrigger CI
- 0f31170 Fix mypy errors from minio 7.2.20 type stubs.
- e3cab5e chore(deps): update dependencies [automated]
- f428a97 Merge pull request 'Fix dependency bot PR creation for Gitea.' (#15) from cursor/fix-dependency-bot-gitea-pr into main
- a3b9912 Fix dependency bot PR creation for Gitea.
- abaa078 Merge pull request 'Fix dependency update bot to use uv lock --upgrade.' (#14) from cursor/fix-dependency-update-bot-uv-lock into main
- 9864f0c Fix dependency update bot to use uv lock --upgrade.
- e39b467 Merge pull request 'Automate releases from PR titles and clean up CI workflows.' (#13) from cursor/pr-title-release-automation into main
- 85c1a64 Sync uv.lock with pyproject.toml version 0.3.1.
- a8491c0 Automate releases from PR titles and clean up CI workflows.
## [0.3.1] - 2026-06-28
### Summary
Add public repository interfaces and subclassable adapter CRUD API.
### Changed
- 2bb361e Merge pull request 'Add public repository interfaces and subclassable adapter CRUD API.' (#11) from cursor/public-repository-api into main
- c5169df Use pre-commit for Prettier in CI and document local hook setup.
- 497533f Fix mypy context manager typing for adapter subclasses.
- e98fd3b Add public repository interfaces and subclassable adapter CRUD API.
- f4280f1 Merge pull request 'Remove non-existent Mongo adapter from README.' (#10) from cleanup/remove-mongo-adapter-mentions into main
- d9b9fa4 updated package settings
- 9d25f7c updated package settings
- 1635815 updated package settings
- 933c4ee updated package settings
- 2b0d98f updated package settings
- 729fa48 Remove non-existent Mongo adapter from README.
## [0.2.3] - 2025-09-18
### Summary
added support for content type when putting object in minio
### Changed
- fa011be Merge pull request 'added support for content type when putting object in minio' (#9) from add-support-for-content-type-in-minio into main
- 2e93309 ruff format fix
- 3ab8da9 added support for content type when putting object in minio
## [0.2.2] - 2025-09-17
### Summary
moved import of optional packages into classes
### Changed
- 1dd604d Merge pull request 'moved import of optional packages into classes' (#6) from make-optional-parts-independent into main
- e5c4025 added graceful handling of missing optional packages
- 396cf83 code quality fixes
- 538b26b mypy fixed
- ee6acc6 moved import of optional packages into classes
## [0.2.1] - 2025-09-15
### Summary
exposed minio adapter
### Changed
- 0ca07c0 Merge pull request 'exposed minio adapter' (#5) from expose-minio-adapter into main
- 7446e88 added py.typed mypy config file
- d5f437f ruff format fix
- 7beca7c exposed minio adapter
## [0.2.0] - 2025-09-15
### Summary
add-minio-adapter
### Changed
- 65d8051 Merge pull request 'add-minio-adapter' (#4) from add-minio-adapter into main
- f5af303 auto install all extra packages
- 3260a3f added new optional package to ci
- c7bced4 ruff lint fixes
- 800d702 pyupgrade
- 5a72507 ruff format
- c45ba3c mypy fixes
- a3b5443 added minio adapter and tests
- cc99fe5 minor corrections
- 93fffe1 added optional group minio
## [0.1.0] - 2025-09-14
### Summary
Release v0.1.0
### Changed
- 2eafd73 Merge pull request 'added redis adapter' (#1) from collect-code-from-other-repos into main
- 1cc047d removed unused bound ports for test
- c7cc2a3 added optional package redis
- b5d0297 pyupgrade fixed imports
- 2fa633a ruff fixed formatting
- 0e47db2 fuxed mypy issues
- 0948799 added integration tests
- 524bd0f fixed connection error handling
- 969c2c8 installed more pytest-cov
- 7caee87 added installing optional redis group
- 918535e added test coverage and upload step
- 5d7afe7 ruff format corrections
- 59a943e added redis types stub and fixed return type
- 1abcae4 added redis adapter
- c789e33 initial commit
- 56cad42 Initial commit
+121 -14
View File
@@ -7,23 +7,30 @@ Subclass an adapter in your own repository to add domain-specific methods while
## Architecture
| Layer | Responsibility |
| ---------------- | ----------------------------------------------------------------- |
| **Interfaces** | Abstract contracts for connection, context, and CRUD |
| **Adapters** | Technology-specific base classes (`RedisAdapter`, `MinioAdapter`) |
| ---------------- | ---------------------------------------------------------------------------------------------------- |
| **Interfaces** | Abstract contracts for connection, context, CRUD, and queues |
| **Adapters** | Technology-specific base classes (`RedisAdapter`, `MinioAdapter`, `PostgresAdapter`, queue adapters) |
| **Your project** | Subclass an adapter and add domain methods |
Each public interface is a `@runtime_checkable` `Protocol` with `@abstractmethod` members. **Subclass an adapter** when you need connection management and shared behavior — explicit subclasses get runtime instantiation guards and inherited default methods (e.g. `scan_keys`). **Type-annotate against an interface** when you want loose coupling — any object with the right methods satisfies the contract for mypy and `isinstance()` checks, without inheriting from this package.
Connection adapters expose `connect()`, `disconnect()`, and `is_connected()`. The latter verifies backend reachability with a cached health probe (default TTL: 1 second). Subclasses may override `health_check_ttl_seconds`. `connect()` is idempotent: calling it while already connected and healthy is a no-op.
## Future direction
The current API is synchronous. Async repository interfaces and adapters may be added in a future release; existing sync usage would remain supported.
## Optional dependencies
Repository **interfaces** import with the base package. **Adapters** require the matching extra; importing an adapter without its extra raises `ImportError` with install instructions.
Repository **interfaces** import with the base package. Networked **adapters** (`RedisAdapter`, `MinioAdapter`, `PostgresAdapter`) require the matching extra; importing one without its extra raises `ImportError` with install instructions. Queue adapters (`MemoryQueueAdapter`, `FileBackedQueueAdapter`) need no extra.
Install with the extras you need:
```bash
uv add python-repositories[redis]
uv add python-repositories[minio]
uv add python-repositories[redis,minio]
uv add python-repositories[postgres]
uv add python-repositories[redis,minio,postgres]
```
### Redis (`JsonRepositoryInterface`)
@@ -57,16 +64,69 @@ for key in repo.scan_keys("user:*"):
| `MINIO_SECURE` | Use HTTPS (`true`/`false`; default: `true`) |
| `MINIO_CREATE_BUCKET_IF_MISSING` | Auto-create `MINIO_BUCKET` on connect (`true`/`false`; default: `false`) |
Copy [`.env.example`](.env.example) to `.env` for local development. `RedisConfig.from_env()` and `MinioConfig.from_env()` load `.env` automatically when resolving configuration from the environment.
For production, it is recommended to leave `MINIO_CREATE_BUCKET_IF_MISSING` unset so that `connect()` fails fast if the expected bucket is missing. For local development, you will often want `MINIO_SECURE=false` and `MINIO_CREATE_BUCKET_IF_MISSING=true`.
### Postgres (`TableRepositoryInterface`)
Requires PostgreSQL 10 or later; tested against PostgreSQL 16 in CI. Tables are owned by your migrations — the adapter verifies the configured table exists on connect.
| Environment variable | Description |
| ---------------------- | --------------------------------------- |
| `POSTGRES_URI` | PostgreSQL connection URL |
| `POSTGRES_TABLE` | Table name for CRUD operations |
| `POSTGRES_PRIMARY_KEY` | Primary key column name (default: `id`) |
### File-backed queue (`QueueRepositoryInterface`)
In-process and disk-backed FIFO queues for buffering dict items. No optional extra required.
| Adapter | Role |
| ------------------------ | --------------------------------------------------------------------------------- |
| `MemoryQueueAdapter` | Thread-safe in-memory buffer with optional dedup and age eviction |
| `FileBackedQueueAdapter` | Mirrors memory to a JSONL file; replays on `connect()`, compacts on dequeue/evict |
| Environment variable | Description |
| -------------------------- | ----------------------------------------- |
| `FILE_QUEUE_PATH` | Path to the JSONL queue file |
| `FILE_QUEUE_MAX_AGE_HOURS` | Retention window in hours (default: `24`) |
Dedup keys and the age field name are domain-specific: pass them when constructing `FileQueueConfig` (or as kwargs to `FileQueueConfig.from_env(...)`). Callers choose the file path and item schema.
```python
from pathlib import Path
from python_repositories import FileBackedQueueAdapter, FileQueueConfig
config = FileQueueConfig(
path=Path("/var/lib/my-service/queue/items.jsonl"),
max_age_hours=24,
dedup_keys=("id",),
age_key="created_at",
)
with FileBackedQueueAdapter(config=config) as queue:
queue.enqueue([{"id": 1, "created_at": "2024-01-01T00:00:00+00:00"}])
batch = queue.dequeue_batch(max_items=100)
```
Copy [`.env.example`](.env.example) to `.env` for local development. `RedisConfig.from_env()`, `MinioConfig.from_env()`, `PostgresConfig.from_env()`, and `FileQueueConfig.from_env()` load `.env` automatically when resolving configuration from the environment.
## Configuration injection
Adapters accept optional `config` and `client` keyword arguments for explicit setup and testing:
Adapters accept optional `config` and `client` keyword arguments for explicit setup and testing. Queue adapters accept `config` (and an optional injected `memory` buffer for `FileBackedQueueAdapter`):
```python
from python_repositories import RedisAdapter, RedisConfig, MinioAdapter, MinioConfig
from pathlib import Path
from python_repositories import (
FileBackedQueueAdapter,
FileQueueConfig,
MinioAdapter,
MinioConfig,
PostgresAdapter,
PostgresConfig,
RedisAdapter,
RedisConfig,
)
redis = RedisAdapter(config=RedisConfig(uri="redis://localhost:6379"))
minio = MinioAdapter(
@@ -79,6 +139,20 @@ minio = MinioAdapter(
# create_bucket_if_missing=True, # convenient for local dev
)
)
postgres = PostgresAdapter(
config=PostgresConfig(
uri="postgresql://localhost/mydb",
table="users",
primary_key="user_id",
)
)
queue = FileBackedQueueAdapter(
config=FileQueueConfig(
path=Path("/var/lib/my-service/queue/items.jsonl"),
dedup_keys=("id",),
age_key="created_at",
)
)
```
When both `config` and `client` are provided, `connect()` skips client creation (the caller owns the client lifecycle). `config` is required whenever `client` is injected.
@@ -91,7 +165,7 @@ from python_repositories import load_dotenv
load_dotenv() # optional — from_env() also loads .env by default
```
Calling `RedisAdapter()` or `MinioAdapter()` with no arguments still loads configuration from environment variables (and `.env` if present).
Calling `RedisAdapter()`, `MinioAdapter()`, or `PostgresAdapter()` with no arguments still loads configuration from environment variables (and `.env` if present).
## Quick start
@@ -120,6 +194,17 @@ with ArtifactObjectRepository() as repo:
data = repo.get_artifact("report-1")
```
### Relational rows with Postgres
```python
from python_repositories.examples.user_table_repository import UserTableRepository
with UserTableRepository() as repo:
repo.save_user("alice", {"name": "Alice", "email": "alice@example.com"})
user = repo.get_user("alice")
repo.delete_user("alice")
```
### Subclassing in your own project
```python
@@ -144,12 +229,19 @@ class UserRepository(RedisAdapter):
from python_repositories import (
ConnectionAwareInterface,
ContextAwareInterface,
FileBackedQueueAdapter,
FileQueueConfig,
JsonRepositoryInterface,
MemoryQueueAdapter,
MinioAdapter,
MinioConfig,
ObjectRepositoryInterface,
PostgresAdapter,
PostgresConfig,
QueueRepositoryInterface,
RedisAdapter,
RedisConfig,
TableRepositoryInterface,
load_dotenv,
)
```
@@ -161,10 +253,22 @@ uv sync --all-extras
uv run pre-commit install # once per clone — runs hooks on git commit
uv run pytest tests/unit/ -v # fast, no Docker
uv run pytest -m "not integration" -v # all non-Docker tests
uv run pytest tests/integration/redis/ -v # Redis container only
uv run pytest tests/integration/minio/ -v # MinIO container only
uv run pytest tests/integration/postgres/ -v # Postgres container only
uv run pytest -v # full suite (requires Docker)
```
Integration tests are marked with `@pytest.mark.integration` and require Docker (testcontainers). Run unit tests alone for quick local feedback.
Integration tests are marked with `@pytest.mark.integration` and require Docker (testcontainers). Backend-specific markers (`needs_redis`, `needs_minio`, `needs_postgres`) let you run only the containers a test module needs. Run unit tests alone for quick local feedback.
### Test organization
Unit tests live in `tests/unit/` and follow a one-to-one naming convention: `<module>_test.py` tests `python_repositories/<module path>.py`. Examples:
- `json_repository_interface.py``tests/unit/json_repository_interface_test.py`
- `redis_adapter.py``tests/unit/redis_adapter_test.py`
Add new tests to the existing file for that module rather than creating cross-cutting test files. Shared fixtures belong in `tests/conftest.py`; module-specific helpers may live in the matching test file.
### CI base image
@@ -194,12 +298,15 @@ uv run pre-commit run --all-files
Releases are automated when a pull request is merged to `main`. CI reads the **merged PR title** to decide whether and how to bump the version.
- **Version history:** [`CHANGELOG.md`](CHANGELOG.md)
- **Gitea releases:** [releases page](https://gitea.lille-vemmelund.dk/LilleVemmelund/python-repositories/releases)
### How it works
1. Open a PR targeting `main` (see [`.gitea/PULL_REQUEST_TEMPLATE.md`](.gitea/PULL_REQUEST_TEMPLATE.md)).
2. If the PR changes files under `python_repositories/`, the title **must** start with a version bump prefix (enforced by CI).
3. On merge, [`release.yml`](.gitea/workflows/release.yml) bumps [`pyproject.toml`](pyproject.toml) and syncs [`uv.lock`](uv.lock), commits, tags `vX.Y.Z`, creates a Gitea release with auto-generated notes, and pushes the tag.
4. [`publish.yml`](.gitea/workflows/publish.yml) builds and publishes the package to the Gitea Package Registry.
3. On merge, [`release.yml`](.gitea/workflows/release.yml) bumps [`pyproject.toml`](pyproject.toml) and syncs [`uv.lock`](uv.lock), updates [`CHANGELOG.md`](CHANGELOG.md), commits, tags `vX.Y.Z`, creates a Gitea release with auto-generated notes, pushes the tag, and publishes the package to the Gitea Package Registry.
4. [`publish.yml`](.gitea/workflows/publish.yml) handles manual `v*.*.*` tag pushes only (automated releases publish from `release.yml` because release commits use `[skip ci]`, which suppresses tag-push workflows).
Docs-, CI-, and test-only PRs do not need a prefix and will not trigger a release.
@@ -219,7 +326,7 @@ Example titles:
### Release notes
Release notes are generated from commits since the previous tag (see [`scripts/ci/generate-release-notes.sh`](scripts/ci/generate-release-notes.sh)).
Release notes are generated from commits since the previous tag (see [`scripts/ci/generate-release-notes.sh`](scripts/ci/generate-release-notes.sh)). The same content is appended to [`CHANGELOG.md`](CHANGELOG.md) on each release (see [`scripts/ci/update-changelog.sh`](scripts/ci/update-changelog.sh)).
### Manual release
+29 -7
View File
@@ -1,6 +1,6 @@
[project]
name = "python-repositories"
version = "2.0.3"
version = "2.2.0"
description = "Various python repository interfaces exposed as a python package."
authors = [
{ name = "Brian Bjarke Jensen", email = "schnitzelen@gmail.com" }
@@ -10,6 +10,7 @@ requires-python = ">=3.12"
license = { text = "MIT" }
classifiers = [
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.12",
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
]
@@ -19,6 +20,11 @@ dependencies = [
"structlog>=25.4.0",
]
[project.urls]
Repository = "https://gitea.lille-vemmelund.dk/LilleVemmelund/python-repositories"
Changelog = "https://gitea.lille-vemmelund.dk/LilleVemmelund/python-repositories/src/branch/main/CHANGELOG.md"
Releases = "https://gitea.lille-vemmelund.dk/LilleVemmelund/python-repositories/releases"
[project.optional-dependencies]
redis = [
"redis>=6.4.0",
@@ -26,6 +32,9 @@ redis = [
minio = [
"minio>=7.2.16",
]
postgres = [
"psycopg[binary]>=3.2.0",
]
[build-system]
requires = ["hatchling"]
@@ -33,10 +42,12 @@ build-backend = "hatchling.build"
[tool.pytest.ini_options]
testpaths = ["tests"]
pythonpath = ["."]
addopts = "--import-mode=importlib"
markers = [
"integration: tests requiring Docker containers (deselect with '-m \"not integration\"')",
"needs_redis: integration test requiring a Redis container",
"needs_minio: integration test requiring a MinIO container",
"needs_postgres: integration test requiring a Postgres container",
]
[tool.coverage.run]
@@ -60,6 +71,18 @@ name = "gitea"
url = "https://gitea.lille-vemmelund.dk/api/packages/brian/pypi/simple/"
explicit = true
[tool.ruff]
line-length = 88
target-version = "py312"
[tool.ruff.lint]
extend-select = ["I"]
[tool.ruff.lint.isort]
known-first-party = ["python_repositories", "tests"]
combine-as-imports = true
force-sort-within-sections = true
[tool.mypy]
python_version = "3.12"
warn_return_any = true # nudge to use stricter types
@@ -87,13 +110,12 @@ ignore_missing_imports = true
[dependency-groups]
dev = [
"mypy>=1.17.1",
"pre-commit>=4.3.0",
"mypy>=2.1.0",
"pre-commit>=4.6.0",
"pytest>=8.4.1",
"pytest-cov>=7.0.0",
"pyupgrade>=3.20.0",
"ruff>=0.12.11",
"pyupgrade>=3.21.2",
"ruff>=0.15.20",
"safety>=3.6.0",
"testcontainers>=4.13.0",
"types-redis>=4.6.0.20241004",
]
+18 -1
View File
@@ -6,26 +6,43 @@ from typing import TYPE_CHECKING
# Interfaces are always available; they have no optional backend dependencies.
from . import adapters
from .config import MinioConfig, RedisConfig, load_dotenv
from .config import (
FileQueueConfig,
MinioConfig,
PostgresConfig,
RedisConfig,
load_dotenv,
)
from .interfaces import (
ConnectionAwareInterface,
ContextAwareInterface,
JsonRepositoryInterface,
ObjectRepositoryInterface,
QueueRepositoryInterface,
TableRepositoryInterface,
)
# Adapters are imported only for static type checkers; runtime loading is delegated below.
if TYPE_CHECKING:
from .adapters.file_backed_queue_adapter import (
FileBackedQueueAdapter as FileBackedQueueAdapter,
)
from .adapters.memory_queue_adapter import MemoryQueueAdapter as MemoryQueueAdapter
from .adapters.minio_adapter import MinioAdapter as MinioAdapter
from .adapters.postgres_adapter import PostgresAdapter as PostgresAdapter
from .adapters.redis_adapter import RedisAdapter as RedisAdapter
__all__ = [
"ConnectionAwareInterface",
"ContextAwareInterface",
"FileQueueConfig",
"JsonRepositoryInterface",
"MinioConfig",
"ObjectRepositoryInterface",
"PostgresConfig",
"QueueRepositoryInterface",
"RedisConfig",
"TableRepositoryInterface",
"load_dotenv",
*adapters.__all__,
]
+12
View File
@@ -11,7 +11,10 @@ from typing import TYPE_CHECKING
# Adapters are imported only for static type checkers; runtime loading is deferred below.
if TYPE_CHECKING:
from .file_backed_queue_adapter import FileBackedQueueAdapter
from .memory_queue_adapter import MemoryQueueAdapter
from .minio_adapter import MinioAdapter
from .postgres_adapter import PostgresAdapter
from .redis_adapter import RedisAdapter
# Map public adapter names to their defining module and class.
@@ -20,11 +23,20 @@ if TYPE_CHECKING:
_LAZY_EXPORTS = {
"RedisAdapter": (".redis_adapter", "RedisAdapter"),
"MinioAdapter": (".minio_adapter", "MinioAdapter"),
"PostgresAdapter": (".postgres_adapter", "PostgresAdapter"),
"MemoryQueueAdapter": (".memory_queue_adapter", "MemoryQueueAdapter"),
"FileBackedQueueAdapter": (
".file_backed_queue_adapter",
"FileBackedQueueAdapter",
),
}
__all__ = [
"RedisAdapter",
"MinioAdapter",
"PostgresAdapter",
"MemoryQueueAdapter",
"FileBackedQueueAdapter",
]
@@ -2,8 +2,8 @@
from __future__ import annotations
import time
from abc import abstractmethod
import time
from typing import Self
import structlog
@@ -0,0 +1,215 @@
"""File-backed queue adapter: in-memory hot buffer mirrored to JSONL on disk."""
from __future__ import annotations
from datetime import UTC, datetime, timedelta
import json
import os
from pathlib import Path
import threading
from typing import Any, Self, TextIO
import structlog
from python_repositories.adapters.memory_queue_adapter import MemoryQueueAdapter
from python_repositories.config.file_queue_config import FileQueueConfig
from python_repositories.interfaces.queue_repository_interface import (
QueueRepositoryInterface,
)
class FileBackedQueueAdapter(QueueRepositoryInterface):
"""Queue that mirrors an in-memory buffer to an append-friendly JSONL file.
Does not extend ``ConnectionAwareAdapter``. ``connect()`` replays the JSONL
file into memory; ``disconnect()`` flushes any open handle.
"""
def __init__(
self,
*,
config: FileQueueConfig | None = None,
memory: MemoryQueueAdapter | None = None,
) -> None:
if config is None:
config = FileQueueConfig.from_env()
if memory is None:
memory = MemoryQueueAdapter(
dedup_keys=config.dedup_keys,
age_key=config.age_key,
)
self._config = config
self._memory = memory
self._lock = threading.RLock()
self._connected = False
self._append_handle: TextIO | None = None
self.logger = structlog.get_logger(self.__class__.__name__)
@staticmethod
def _json_default(value: object) -> str:
if isinstance(value, datetime):
return value.isoformat()
raise TypeError(f"Object of type {type(value)!r} is not JSON serializable")
def __enter__(self) -> Self:
self.connect()
return self
def __exit__(
self, exc_type: type | None, exc_val: object | None, exc_tb: object | None
) -> None:
del exc_type, exc_val, exc_tb
self.disconnect()
def connect(self) -> None:
"""Create parent directories and replay the JSONL file into memory."""
with self._lock:
if self._connected:
return
path = self._config.path
path.parent.mkdir(parents=True, exist_ok=True)
self._memory.clear()
if path.is_file():
self._replay_file(path)
self._append_handle = path.open("a", encoding="utf-8")
self._connected = True
def disconnect(self) -> None:
"""Flush and close the append handle if open."""
with self._lock:
if self._append_handle is not None:
self._append_handle.flush()
try:
os.fsync(self._append_handle.fileno())
except OSError:
pass
self._append_handle.close()
self._append_handle = None
self._connected = False
def is_connected(self) -> bool:
"""Return whether ``connect()`` has completed successfully."""
return self._connected
def enqueue(self, items: list[dict[str, Any]]) -> None:
"""Append items to memory and the JSONL file; optionally age-evict."""
self._require_connected()
with self._lock:
added = self._memory.enqueue_and_return_added(items)
if added:
self._append_items(added)
if self._config.age_key is not None:
cutoff = datetime.now(UTC) - timedelta(hours=self._config.max_age_hours)
if self._memory.evict_older_than(cutoff):
self._compact()
def dequeue_batch(self, *, max_items: int = 1000) -> list[dict[str, Any]]:
"""Dequeue from memory and compact the JSONL file to match."""
self._require_connected()
with self._lock:
batch = self._memory.dequeue_batch(max_items=max_items)
if batch:
self._compact()
return batch
def size(self) -> int:
"""Return the number of items currently in the in-memory buffer."""
self._require_connected()
return self._memory.size()
def evict_older_than(self, cutoff: datetime) -> int:
"""Evict aged items from memory and compact the JSONL file."""
self._require_connected()
with self._lock:
removed = self._memory.evict_older_than(cutoff)
if removed:
self._compact()
return removed
def _require_connected(self) -> None:
if not self._connected:
raise RuntimeError("FileBackedQueueAdapter is not connected")
def _replay_file(self, path: Path) -> None:
cutoff: datetime | None = None
if self._config.age_key is not None:
cutoff = datetime.now(UTC) - timedelta(hours=self._config.max_age_hours)
with path.open(encoding="utf-8") as handle:
for line_number, line in enumerate(handle, start=1):
stripped = line.strip()
if not stripped:
continue
try:
payload = json.loads(stripped)
except json.JSONDecodeError:
self.logger.warning(
"Skipping corrupt JSONL line",
path=str(path),
line_number=line_number,
)
continue
if not isinstance(payload, dict):
self.logger.warning(
"Skipping non-object JSONL line",
path=str(path),
line_number=line_number,
)
continue
item: dict[str, Any] = payload
if cutoff is not None and self._config.age_key is not None:
age_value = item.get(self._config.age_key)
if age_value is None:
self.logger.warning(
"Skipping item missing age key on replay",
path=str(path),
line_number=line_number,
age_key=self._config.age_key,
)
continue
try:
if MemoryQueueAdapter._parse_age(age_value) < cutoff:
continue
except ValueError:
self.logger.warning(
"Skipping item with invalid age on replay",
path=str(path),
line_number=line_number,
)
continue
try:
self._memory.enqueue_and_return_added([item])
except ValueError as exc:
self.logger.warning(
"Skipping invalid item on replay",
path=str(path),
line_number=line_number,
error=str(exc),
)
def _append_items(self, items: list[dict[str, Any]]) -> None:
if self._append_handle is None:
raise RuntimeError("append handle is not open")
for item in items:
self._append_handle.write(
json.dumps(item, default=self._json_default, separators=(",", ":"))
)
self._append_handle.write("\n")
self._append_handle.flush()
def _compact(self) -> None:
"""Rewrite the JSONL file from the current in-memory snapshot."""
path = self._config.path
tmp_path = path.with_suffix(path.suffix + ".tmp")
if self._append_handle is not None:
self._append_handle.flush()
self._append_handle.close()
self._append_handle = None
with tmp_path.open("w", encoding="utf-8") as handle:
for item in self._memory.snapshot():
handle.write(
json.dumps(item, default=self._json_default, separators=(",", ":"))
)
handle.write("\n")
handle.flush()
tmp_path.replace(path)
self._append_handle = path.open("a", encoding="utf-8")
@@ -0,0 +1,128 @@
"""In-memory queue adapter with optional dedup and age-based eviction."""
from __future__ import annotations
from collections import OrderedDict
from collections.abc import Hashable
from datetime import UTC, datetime
import threading
from typing import Any
from python_repositories.interfaces.queue_repository_interface import (
QueueRepositoryInterface,
)
class MemoryQueueAdapter(QueueRepositoryInterface):
"""Thread-safe in-process FIFO queue with optional deduplication."""
def __init__(
self,
*,
dedup_keys: tuple[str, ...] = (),
age_key: str | None = None,
) -> None:
self._dedup_keys = dedup_keys
self._age_key = age_key
self._lock = threading.RLock()
self._items: OrderedDict[Hashable, dict[str, Any]] = OrderedDict()
self._seq = 0
@staticmethod
def _parse_age(value: object) -> datetime:
"""Normalize an age field to a timezone-aware UTC datetime."""
if isinstance(value, datetime):
if value.tzinfo is None:
return value.replace(tzinfo=UTC)
return value.astimezone(UTC)
if isinstance(value, str):
normalized = value.replace("Z", "+00:00")
parsed = datetime.fromisoformat(normalized)
if parsed.tzinfo is None:
return parsed.replace(tzinfo=UTC)
return parsed.astimezone(UTC)
raise ValueError(
f"age value must be datetime or ISO-8601 str, got {type(value)!r}"
)
def enqueue(self, items: list[dict[str, Any]]) -> None:
"""Append items; skip duplicates when ``dedup_keys`` is configured."""
self.enqueue_and_return_added(items)
def enqueue_and_return_added(
self, items: list[dict[str, Any]]
) -> list[dict[str, Any]]:
"""Enqueue items and return the subset that was newly stored."""
if not items:
return []
added: list[dict[str, Any]] = []
with self._lock:
for raw in items:
item = self._normalize_item(raw)
key = self._make_key(item)
if self._dedup_keys and key in self._items:
continue
self._items[key] = item
added.append(item)
return added
def dequeue_batch(self, *, max_items: int = 1000) -> list[dict[str, Any]]:
"""Remove and return up to ``max_items`` items in FIFO order."""
if max_items < 0:
raise ValueError("max_items must be >= 0")
with self._lock:
batch: list[dict[str, Any]] = []
for _ in range(min(max_items, len(self._items))):
_key, item = self._items.popitem(last=False)
batch.append(item)
return batch
def size(self) -> int:
"""Return the number of items currently in the queue."""
with self._lock:
return len(self._items)
def evict_older_than(self, cutoff: datetime) -> int:
"""Remove items whose age field is strictly older than ``cutoff``."""
if self._age_key is None:
return 0
cutoff_utc = self._parse_age(cutoff)
removed = 0
with self._lock:
to_remove = [
key
for key, item in self._items.items()
if self._parse_age(item[self._age_key]) < cutoff_utc
]
for key in to_remove:
del self._items[key]
removed += 1
return removed
def clear(self) -> None:
"""Remove all items from the queue."""
with self._lock:
self._items.clear()
def snapshot(self) -> list[dict[str, Any]]:
"""Return a shallow copy of queued items in FIFO order."""
with self._lock:
return [dict(item) for item in self._items.values()]
def _normalize_item(self, raw: dict[str, Any]) -> dict[str, Any]:
item = dict(raw)
if self._dedup_keys:
missing = [key for key in self._dedup_keys if key not in item]
if missing:
raise ValueError(f"item missing dedup key(s): {missing}")
if self._age_key is not None:
if self._age_key not in item:
raise ValueError(f"item missing age key: {self._age_key!r}")
item[self._age_key] = self._parse_age(item[self._age_key])
return item
def _make_key(self, item: dict[str, Any]) -> Hashable:
if not self._dedup_keys:
self._seq += 1
return self._seq
return tuple(item[key] for key in self._dedup_keys)
+16 -5
View File
@@ -1,6 +1,7 @@
"""Definition of MinioAdapter class."""
from __future__ import annotations
from io import BytesIO
from python_repositories.adapters.connection_aware_adapter import (
@@ -18,7 +19,7 @@ except ImportError as exc:
) from exc
class MinioAdapter(ObjectRepositoryInterface, ConnectionAwareAdapter):
class MinioAdapter(ConnectionAwareAdapter, ObjectRepositoryInterface):
"""Minio adapter exposing basic CRUD functionality."""
endpoint_env_var_name: str = "MINIO_ENDPOINT"
@@ -113,12 +114,17 @@ class MinioAdapter(ObjectRepositoryInterface, ConnectionAwareAdapter):
data: BytesIO,
content_type: str = "application/octet-stream",
) -> None:
"""Put an object into the Minio bucket."""
"""Put an object into the Minio bucket.
Accepts zero-byte ``BytesIO``. A zero-byte object is returned by
``get()`` as an empty buffer, not ``None``. Use ``delete()`` to remove
an object entirely.
"""
# Check input
if not isinstance(object_name, str) or len(object_name) == 0:
raise ValueError("object_name must be a non-empty string")
if not isinstance(data, BytesIO) or data.getbuffer().nbytes == 0:
raise ValueError("data must be a non-empty BytesIO object")
if not isinstance(data, BytesIO):
raise ValueError("data must be a BytesIO object")
if not isinstance(content_type, str) or len(content_type) == 0:
raise ValueError("content_type must be a non-empty string")
# Check connection
@@ -142,7 +148,12 @@ class MinioAdapter(ObjectRepositoryInterface, ConnectionAwareAdapter):
)
def get(self, object_name: str) -> BytesIO | None:
"""Get an object from the Minio bucket."""
"""Get an object from the Minio bucket.
Returns ``None`` when the object does not exist. Returns an empty
``BytesIO`` for a zero-byte object. Use ``value is not None`` to test
existence.
"""
# Check input
if not isinstance(object_name, str) or len(object_name) == 0:
raise ValueError("object_name must be a non-empty string")
@@ -0,0 +1,230 @@
"""Definition of PostgresAdapter class."""
from __future__ import annotations
from typing import Any
from python_repositories.adapters.connection_aware_adapter import (
ConnectionAwareAdapter,
)
from python_repositories.config import PostgresConfig
from python_repositories.interfaces import TableRepositoryInterface
try:
import psycopg
from psycopg import sql
from psycopg.errors import UndefinedTable
from psycopg.rows import dict_row
except ImportError as exc:
raise ImportError(
"Postgres support requires the postgres extra. "
"Install with: pip install python-repositories[postgres]"
) from exc
class PostgresAdapter(ConnectionAwareAdapter, TableRepositoryInterface):
"""Postgres adapter exposing basic table CRUD functionality."""
uri_env_var_name: str = "POSTGRES_URI"
table_env_var_name: str = "POSTGRES_TABLE"
primary_key_env_var_name: str = "POSTGRES_PRIMARY_KEY"
connection_name: str = "Postgres"
def __init__(
self,
*,
config: PostgresConfig | None = None,
client: psycopg.Connection[Any] | None = None,
) -> None:
super().__init__()
if client is not None and config is None:
raise ValueError("config is required when client is provided")
if config is None:
config = PostgresConfig.from_env(
self.uri_env_var_name,
self.table_env_var_name,
self.primary_key_env_var_name,
)
self._config = config
self._client_injected = client is not None
self._client: psycopg.Connection[Any] | None = client
if self._client_injected:
self._invalidate_health_cache()
def _is_client_ready(self) -> bool:
return self._client is not None
def _table_identifier(self) -> sql.Identifier:
return sql.Identifier(self._config.table)
def _primary_key_identifier(self) -> sql.Identifier:
return sql.Identifier(self._config.primary_key)
def _validate_injected_client(self) -> None:
if self._client is None:
return
try:
self._run_connection_probe()
except (psycopg.Error, ConnectionError) as exc:
raise ConnectionError(
f"Could not connect to Postgres at {self._config.uri}"
) from exc
def _establish_connection(self) -> None:
uri = self._config.uri
try:
client = psycopg.connect(
uri,
row_factory=dict_row,
connect_timeout=10,
autocommit=True,
)
self._client = client
self._run_connection_probe()
except ConnectionError:
self._client = None
raise
except psycopg.Error as exc:
self._client = None
raise ConnectionError(f"Could not connect to Postgres at {uri}") from exc
def _run_connection_probe(self) -> None:
assert self._client is not None
with self._client.cursor() as cursor:
cursor.execute("SELECT 1")
try:
cursor.execute(
sql.SQL("SELECT 1 FROM {} LIMIT 0").format(self._table_identifier())
)
except UndefinedTable as exc:
raise ConnectionError(
f"Table '{self._config.table}' does not exist on Postgres at "
f"{self._config.uri}"
) from exc
def disconnect(self) -> None:
"""Disconnect from the Postgres server."""
if self._client is not None and not self._client_injected:
self._client.close()
self._client = None
self._invalidate_health_cache()
def _probe_connection(self) -> bool:
assert self._client is not None
try:
self._run_connection_probe()
except (psycopg.Error, ConnectionError):
return False
return True
def _validate_pk(self, pk: Any) -> None:
if pk is None:
raise ValueError("Primary key must not be None")
def _validate_row(self, row: dict[str, Any]) -> None:
if not isinstance(row, dict):
raise ValueError("Row must be a dictionary")
if self._config.primary_key not in row:
raise ValueError(
f"Row must include primary key column '{self._config.primary_key}'"
)
def _validate_limit(self, limit: int | None) -> None:
if limit is not None and (not isinstance(limit, int) or limit < 0):
raise ValueError("Limit must be a non-negative integer or None")
def fetch_one(self, pk: Any) -> dict[str, Any] | None:
"""Fetch a single row by primary key."""
self._validate_pk(pk)
self._require_connected()
assert self._client is not None
query = sql.SQL("SELECT * FROM {} WHERE {} = %s").format(
self._table_identifier(),
self._primary_key_identifier(),
)
with self._client.cursor() as cursor:
cursor.execute(query, (pk,))
row = cursor.fetchone()
self.logger.debug("Fetched row", pk=pk, found=row is not None)
return row
def fetch_all(self, *, limit: int | None = None) -> list[dict[str, Any]]:
"""Fetch all rows from the configured table."""
self._validate_limit(limit)
self._require_connected()
assert self._client is not None
query = sql.SQL("SELECT * FROM {}").format(self._table_identifier())
params: tuple[Any, ...] = ()
if limit is not None:
query = sql.Composed([query, sql.SQL(" LIMIT %s")])
params = (limit,)
with self._client.cursor() as cursor:
cursor.execute(query, params)
rows = cursor.fetchall()
self.logger.debug("Fetched rows", count=len(rows), limit=limit)
return list(rows)
def upsert(self, row: dict[str, Any]) -> None:
"""Insert or update a row by primary key."""
self._validate_row(row)
self._require_connected()
assert self._client is not None
pk_col = self._config.primary_key
columns = list(row.keys())
identifiers = [sql.Identifier(column) for column in columns]
placeholders = sql.SQL(", ").join(sql.Placeholder() * len(columns))
column_list = sql.SQL(", ").join(identifiers)
update_columns = [column for column in columns if column != pk_col]
on_conflict: sql.Composed | sql.SQL
if update_columns:
update_assignments = sql.SQL(", ").join(
sql.SQL("{} = EXCLUDED.{}").format(
sql.Identifier(column),
sql.Identifier(column),
)
for column in update_columns
)
on_conflict = sql.SQL("DO UPDATE SET {}").format(update_assignments)
else:
on_conflict = sql.SQL("DO NOTHING")
query = sql.SQL("INSERT INTO {} ({}) VALUES ({}) ON CONFLICT ({}) {}").format(
self._table_identifier(),
column_list,
placeholders,
self._primary_key_identifier(),
on_conflict,
)
with self._client.cursor() as cursor:
cursor.execute(query, tuple(row[column] for column in columns))
self.logger.debug("Upserted row", pk=row[pk_col], columns=columns)
def delete(self, pk: Any) -> None:
"""Delete a row by primary key."""
self._validate_pk(pk)
self._require_connected()
assert self._client is not None
query = sql.SQL("DELETE FROM {} WHERE {} = %s").format(
self._table_identifier(),
self._primary_key_identifier(),
)
with self._client.cursor() as cursor:
cursor.execute(query, (pk,))
self.logger.debug("Deleted row", pk=pk)
def execute(
self,
sql_text: str,
params: tuple[Any, ...] = (),
) -> list[dict[str, Any]]:
"""Run a read SQL statement and return rows as dicts."""
if not isinstance(sql_text, str) or len(sql_text) == 0:
raise ValueError("SQL must be a non-empty string")
if not isinstance(params, tuple):
raise ValueError("Params must be a tuple")
self._require_connected()
assert self._client is not None
with self._client.cursor() as cursor:
cursor.execute(sql_text, params)
rows = cursor.fetchall()
self.logger.debug("Executed SQL", row_count=len(rows))
return list(rows)
+16 -5
View File
@@ -1,6 +1,7 @@
"""Definition of RedisAdapter class."""
from __future__ import annotations
from collections.abc import Iterator
from typing import Any, cast
@@ -20,7 +21,7 @@ except ImportError as exc:
) from exc
class RedisAdapter(JsonRepositoryInterface, ConnectionAwareAdapter):
class RedisAdapter(ConnectionAwareAdapter, JsonRepositoryInterface):
"""Redis adapter exposing basic CRUD functionality."""
uri_env_var_name: str = "REDIS_URI"
@@ -90,12 +91,17 @@ class RedisAdapter(JsonRepositoryInterface, ConnectionAwareAdapter):
return False
def set(self, key: str, data: dict[str, Any]) -> None:
"""Set a JSON object in Redis."""
"""Set a JSON object in Redis.
Accepts any dict, including ``{}``. An empty dict creates a key that
``get()`` returns as ``{}``, not ``None``. Use ``delete()`` to remove a
key entirely.
"""
# Check input
if not isinstance(key, str) or len(key) == 0:
raise ValueError("Key must be a non-empty string")
if not isinstance(data, dict) or len(data) == 0:
raise ValueError("Data must be a non-empty dictionary")
if not isinstance(data, dict):
raise ValueError("Data must be a dictionary")
# Check connection
self._require_connected()
assert self._client is not None
@@ -104,7 +110,12 @@ class RedisAdapter(JsonRepositoryInterface, ConnectionAwareAdapter):
self.logger.debug("Set key", key=key, data_keys=list(data.keys()))
def get(self, key: str) -> dict[str, Any] | None:
"""Get a JSON object from Redis."""
"""Get a JSON object from Redis.
Returns ``None`` if the key is absent. Returns ``{}`` if the key exists
with an empty JSON object. Use ``value is not None`` to test existence;
avoid truthiness checks (``{}`` is falsy).
"""
# Check input
if not isinstance(key, str) or len(key) == 0:
raise ValueError("Key must be a non-empty string")
+4
View File
@@ -1,9 +1,13 @@
from .dotenv_loader import load_dotenv as load_dotenv
from .file_queue_config import FileQueueConfig as FileQueueConfig
from .minio_config import MinioConfig as MinioConfig
from .postgres_config import PostgresConfig as PostgresConfig
from .redis_config import RedisConfig as RedisConfig
__all__ = [
"FileQueueConfig",
"MinioConfig",
"PostgresConfig",
"RedisConfig",
"load_dotenv",
]
@@ -0,0 +1,56 @@
"""File-backed queue configuration."""
from __future__ import annotations
from dataclasses import dataclass
import os
from pathlib import Path
from python_utils import check_env
from python_repositories.config.dotenv_loader import load_dotenv
_DEFAULT_MAX_AGE_HOURS = 24
@dataclass(frozen=True)
class FileQueueConfig:
"""Configuration for a JSONL file-backed queue.
Path and key schema are owned by the caller. This package does not assume
any particular directory layout or item field names.
"""
path: Path
max_age_hours: int = _DEFAULT_MAX_AGE_HOURS
dedup_keys: tuple[str, ...] = ()
age_key: str | None = None
@classmethod
def from_env(
cls,
path_env_var_name: str = "FILE_QUEUE_PATH",
*,
max_age_hours_env_var_name: str = "FILE_QUEUE_MAX_AGE_HOURS",
dedup_keys: tuple[str, ...] = (),
age_key: str | None = None,
use_dotenv: bool = True,
) -> FileQueueConfig:
"""Load path and optional max age from environment variables.
``dedup_keys`` and ``age_key`` are domain-specific and must be passed
explicitly; they are not read from the environment.
"""
if use_dotenv:
load_dotenv()
check_env(path_env_var_name)
raw_max_age = os.getenv(max_age_hours_env_var_name)
max_age_hours = (
_DEFAULT_MAX_AGE_HOURS if raw_max_age is None else int(raw_max_age)
)
return cls(
path=Path(str(os.getenv(path_env_var_name))),
max_age_hours=max_age_hours,
dedup_keys=dedup_keys,
age_key=age_key,
)
+1 -1
View File
@@ -2,8 +2,8 @@
from __future__ import annotations
import os
from dataclasses import dataclass
import os
from python_utils import check_env
@@ -0,0 +1,42 @@
"""PostgreSQL connection configuration."""
from __future__ import annotations
from dataclasses import dataclass
import os
from python_utils import check_env
from python_repositories.config.dotenv_loader import load_dotenv
@dataclass(frozen=True)
class PostgresConfig:
"""Configuration for connecting to PostgreSQL."""
uri: str
table: str
primary_key: str = "id"
@classmethod
def from_env(
cls,
uri_env_var_name: str = "POSTGRES_URI",
table_env_var_name: str = "POSTGRES_TABLE",
primary_key_env_var_name: str = "POSTGRES_PRIMARY_KEY",
*,
use_dotenv: bool = True,
) -> PostgresConfig:
"""Load configuration from environment variables."""
if use_dotenv:
load_dotenv()
env_var_names = {uri_env_var_name, table_env_var_name}
check_env(env_var_names)
primary_key = os.getenv(primary_key_env_var_name)
if primary_key is None or primary_key == "":
primary_key = "id"
return cls(
uri=str(os.getenv(uri_env_var_name)),
table=str(os.getenv(table_env_var_name)),
primary_key=primary_key,
)
+1 -1
View File
@@ -2,8 +2,8 @@
from __future__ import annotations
import os
from dataclasses import dataclass
import os
from python_utils import check_env
@@ -0,0 +1,19 @@
"""Example domain repository backed by Postgres tables."""
from typing import Any
from python_repositories.adapters.postgres_adapter import PostgresAdapter
class UserTableRepository(PostgresAdapter):
"""Example: domain repository backed by a Postgres table."""
def get_user(self, user_id: str) -> dict[str, Any] | None:
return self.fetch_one(user_id)
def save_user(self, user_id: str, user: dict[str, Any]) -> None:
row = {self._config.primary_key: user_id, **user}
self.upsert(row)
def delete_user(self, user_id: str) -> None:
self.delete(user_id)
@@ -8,10 +8,18 @@ from .json_repository_interface import (
from .object_repository_interface import (
ObjectRepositoryInterface as ObjectRepositoryInterface,
)
from .queue_repository_interface import (
QueueRepositoryInterface as QueueRepositoryInterface,
)
from .table_repository_interface import (
TableRepositoryInterface as TableRepositoryInterface,
)
__all__ = [
"ConnectionAwareInterface",
"ContextAwareInterface",
"JsonRepositoryInterface",
"ObjectRepositoryInterface",
"QueueRepositoryInterface",
"TableRepositoryInterface",
]
@@ -1,9 +1,11 @@
"""Definition of ConnectionAwareInterface abstract base class."""
"""Definition of ConnectionAwareInterface protocol and abstract base class."""
from abc import ABC, abstractmethod
from abc import abstractmethod
from typing import Protocol, runtime_checkable
class ConnectionAwareInterface(ABC):
@runtime_checkable
class ConnectionAwareInterface(Protocol):
"""Interface that defines connection-related methods."""
@abstractmethod
@@ -1,12 +1,13 @@
"""Definition of ContextAwareInterface abstract base class."""
"""Definition of ContextAwareInterface protocol and abstract base class."""
from __future__ import annotations
from abc import ABC, abstractmethod
from typing import Self
from abc import abstractmethod
from typing import Protocol, Self, runtime_checkable
class ContextAwareInterface(ABC):
@runtime_checkable
class ContextAwareInterface(Protocol):
"""Interface that defines context-related methods."""
@abstractmethod
@@ -1,26 +1,37 @@
"""Definition of JsonRepositoryInterface abstract base class."""
"""Definition of JsonRepositoryInterface protocol and abstract base class."""
from abc import abstractmethod
from collections.abc import Iterator
from abc import ABC, abstractmethod
from typing import Any
from typing import Any, Protocol, runtime_checkable
class JsonRepositoryInterface(ABC):
@runtime_checkable
class JsonRepositoryInterface(Protocol):
"""Interface that defines JSON document CRUD methods."""
@abstractmethod
def get(self, key: str) -> dict[str, Any] | None:
"""Get a JSON object by key."""
"""Get a JSON object by key.
Returns ``None`` when the key is absent. Returns ``{}`` when the key
exists with an empty JSON object. Use ``value is not None`` to test
existence; avoid truthiness checks (``{}`` is falsy).
"""
...
@abstractmethod
def set(self, key: str, data: dict[str, Any]) -> None:
"""Set a JSON object by key."""
"""Set a JSON object by key.
Accepts any dict, including ``{}``. An empty dict creates a key that
``get()`` returns as ``{}``, not ``None``. Use ``delete()`` to remove a
key entirely.
"""
...
@abstractmethod
def delete(self, key: str) -> None:
"""Delete a JSON object by key."""
"""Delete a JSON object by key, removing it entirely."""
...
@abstractmethod
@@ -1,18 +1,22 @@
"""Definition of ObjectRepositoryInterface abstract base class."""
"""Definition of ObjectRepositoryInterface protocol and abstract base class."""
from abc import ABC, abstractmethod
from abc import abstractmethod
from io import BytesIO
from typing import Protocol, runtime_checkable
class ObjectRepositoryInterface(ABC):
@runtime_checkable
class ObjectRepositoryInterface(Protocol):
"""Interface that defines binary object CRUD methods."""
@abstractmethod
def get(self, object_name: str) -> BytesIO | None:
"""Get an object by name.
Returns None when the object does not exist. Raises ConnectionError when
not connected. Other backend errors propagate to the caller.
Returns ``None`` when the object does not exist. Returns an empty
``BytesIO`` for a zero-byte object. Use ``value is not None`` to test
existence. Raises ConnectionError when not connected. Other backend
errors propagate to the caller.
"""
...
@@ -23,12 +27,17 @@ class ObjectRepositoryInterface(ABC):
data: BytesIO,
content_type: str = "application/octet-stream",
) -> None:
"""Put an object by name."""
"""Put an object by name.
Accepts zero-byte ``BytesIO``. A zero-byte object is returned by
``get()`` as an empty buffer, not ``None``. Use ``delete()`` to remove
an object entirely.
"""
...
@abstractmethod
def delete(self, object_name: str) -> None:
"""Delete an object by name."""
"""Delete an object by name, removing it entirely."""
...
@abstractmethod
@@ -0,0 +1,41 @@
"""Definition of QueueRepositoryInterface protocol."""
from abc import abstractmethod
from datetime import datetime
from typing import Any, Protocol, runtime_checkable
@runtime_checkable
class QueueRepositoryInterface(Protocol):
"""Interface that defines buffered queue enqueue/dequeue methods."""
@abstractmethod
def enqueue(self, items: list[dict[str, Any]]) -> None:
"""Append items to the queue.
Duplicates may be skipped when the implementation is configured with
dedup keys. An empty list is a no-op.
"""
...
@abstractmethod
def dequeue_batch(self, *, max_items: int = 1000) -> list[dict[str, Any]]:
"""Remove and return up to ``max_items`` items in FIFO order.
Returns an empty list when the queue is empty.
"""
...
@abstractmethod
def size(self) -> int:
"""Return the number of items currently in the queue."""
...
@abstractmethod
def evict_older_than(self, cutoff: datetime) -> int:
"""Remove items older than ``cutoff`` and return how many were removed.
Age is determined by an implementation-specific item field. When no age
field is configured, this is a no-op that returns ``0``.
"""
...
@@ -0,0 +1,55 @@
"""Definition of TableRepositoryInterface protocol."""
from abc import abstractmethod
from typing import Any, Protocol, runtime_checkable
@runtime_checkable
class TableRepositoryInterface(Protocol):
"""Interface that defines relational table CRUD methods."""
@abstractmethod
def fetch_one(self, pk: Any) -> dict[str, Any] | None:
"""Fetch a single row by primary key.
Returns ``None`` when no row matches. Use ``value is not None`` to test
existence; avoid truthiness checks.
"""
...
@abstractmethod
def fetch_all(self, *, limit: int | None = None) -> list[dict[str, Any]]:
"""Fetch all rows from the configured table.
Returns an empty list when the table has no rows.
"""
...
@abstractmethod
def upsert(self, row: dict[str, Any]) -> None:
"""Insert or update a row by primary key.
``row`` must include the configured primary key column. Raises
``ValueError`` for a non-dict row or a missing primary key.
"""
...
@abstractmethod
def delete(self, pk: Any) -> None:
"""Delete a row by primary key.
Idempotent: no error when the row is already absent.
"""
...
@abstractmethod
def execute(
self,
sql: str,
params: tuple[Any, ...] = (),
) -> list[dict[str, Any]]:
"""Run a read SQL statement and return rows as dicts.
Escape hatch for joins, filters, and other queries in subclasses.
"""
...
+64
View File
@@ -0,0 +1,64 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# shellcheck source=common.sh
source "${SCRIPT_DIR}/common.sh"
CHANGELOG="CHANGELOG.md"
# Best-effort summary for historical releases without stored PR titles.
summary_for_range() {
local from_ref="$1"
local to_ref="$2"
local subject
while IFS= read -r subject; do
[[ -z "$subject" ]] && continue
if echo "$subject" | grep -qE '^chore: release v'; then
continue
fi
if echo "$subject" | grep -qE "^Merge pull request '"; then
local title
title=$(extract_pr_title_from_merge_commit "$subject")
if [[ -n "$title" ]]; then
strip_bump_prefix "$title"
return 0
fi
fi
echo "$subject"
return 0
done < <(git log "${from_ref}..${to_ref}" --pretty=format:'%s' 2>/dev/null || true)
echo "Release ${to_ref}"
}
TAGS=()
while IFS= read -r tag; do
TAGS+=("$tag")
done < <(git tag -l 'v*' --sort=version:refname)
if [[ ${#TAGS[@]} -eq 0 ]]; then
echo "No version tags found." >&2
exit 1
fi
changelog_header > "$CHANGELOG"
for ((i = ${#TAGS[@]} - 1; i >= 0; i--)); do
TAG="${TAGS[$i]}"
VERSION="${TAG#v}"
DATE=$(git log -1 --format=%cs "$TAG" 2>/dev/null || date -u +%Y-%m-%d)
if [[ $i -eq 0 ]]; then
PREV_TAG=""
else
PREV_TAG="${TAGS[$((i - 1))]}"
fi
SUMMARY=$(summary_for_range "$PREV_TAG" "$TAG")
COMMITS=$(commits_between "$PREV_TAG" "$TAG")
SECTION=$(changelog_section "$VERSION" "$DATE" "$SUMMARY" "$COMMITS")
printf '\n%s\n' "$SECTION" >> "$CHANGELOG"
done
echo "Generated ${CHANGELOG} from ${#TAGS[@]} tags." >&2
+57
View File
@@ -61,6 +61,63 @@ set_pyproject_version() {
rm -f pyproject.toml.bak
}
# Returns markdown bullet list of commits between two refs (exclusive..inclusive).
# Optional third argument limits entries when from_ref is empty (release notes only).
commits_between() {
local from_ref="$1"
local to_ref="${2:-HEAD}"
local limit="${3:-}"
if [[ -z "$from_ref" ]]; then
if [[ -n "$limit" ]]; then
git log "${to_ref}" --pretty=format:'- %h %s' -"${limit}" 2>/dev/null || true
else
git log "${to_ref}" --pretty=format:'- %h %s' 2>/dev/null || true
fi
else
git log "${from_ref}..${to_ref}" --pretty=format:'- %h %s' 2>/dev/null || true
fi
}
commits_since() {
commits_between "$1" HEAD
}
changelog_header() {
cat <<'EOF'
# Changelog
All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased]
EOF
}
changelog_section() {
local version="$1"
local date="$2"
local summary="$3"
local commits="$4"
# Blank lines after ATX headings match Prettier v4 markdown formatting
# (see mirrors-prettier in .pre-commit-config.yaml).
cat <<EOF
## [${version}] - ${date}
### Summary
${summary}
### Changed
${commits:-- (no commits recorded)}
EOF
}
bump_semver() {
local current="$1"
local bump_type="$2"
+9 -3
View File
@@ -11,15 +11,16 @@ set -euo pipefail
BRANCH="${BRANCH:-renovate/auto-deps-update}"
BASE_BRANCH="${BASE_BRANCH:-main}"
LOCKFILE="${LOCKFILE:-uv.lock}"
PRE_COMMIT_CONFIG="${PRE_COMMIT_CONFIG:-.pre-commit-config.yaml}"
if git diff --quiet "$LOCKFILE"; then
if git diff --quiet "$LOCKFILE" && git diff --quiet "$PRE_COMMIT_CONFIG"; then
echo "No dependency updates available."
exit 0
fi
git config user.name "CI Bot"
git config user.email "ci-bot@example.com"
git add "$LOCKFILE"
git add "$LOCKFILE" "$PRE_COMMIT_CONFIG"
git commit -m "chore(deps): update dependencies [automated]"
git remote set-url origin "https://x-access-token:${CI_RUNNER_TOKEN}@${GITHUB_SERVER_URL#https://}/${GITHUB_REPOSITORY}.git"
@@ -48,7 +49,12 @@ print(
json.dumps(
{
"title": "chore(deps): update dependencies",
"body": "This PR was created automatically to update dependencies.",
"body": (
"This PR was created automatically to update dependencies.\n\n"
"- `uv.lock`: upgraded Python dependencies\n"
"- `.pre-commit-config.yaml`: updated remote hook revs via "
"`pre-commit autoupdate`"
),
"head": os.environ["BRANCH"],
"base": os.environ["BASE_BRANCH"],
}
+2 -2
View File
@@ -34,13 +34,13 @@ if [[ -n "$PREV_TAG" ]]; then
## Changes since ${PREV_TAG}
$(git log "${PREV_TAG}..HEAD" --pretty=format:'- %h %s' || true)"
$(commits_since "$PREV_TAG")"
else
NOTES="${NOTES}
## Changes
$(git log --pretty=format:'- %h %s' -20 || true)"
$(commits_between "" HEAD 20)"
fi
if [[ -n "${GITHUB_OUTPUT:-}" ]]; then
+63
View File
@@ -0,0 +1,63 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# shellcheck source=common.sh
source "${SCRIPT_DIR}/common.sh"
VERSION="${1:-}"
SUMMARY="${2:-}"
PREV_TAG="${3:-}"
DATE="${4:-$(date -u +%Y-%m-%d)}"
CHANGELOG="CHANGELOG.md"
if [[ -z "$VERSION" ]]; then
echo "Usage: update-changelog.sh <version> <summary> <prev-tag> [date]" >&2
exit 1
fi
if [[ -f "$CHANGELOG" ]] && grep -qF "## [${VERSION}]" "$CHANGELOG"; then
echo "CHANGELOG already contains version ${VERSION}; skipping." >&2
exit 0
fi
if [[ -n "$SUMMARY" ]]; then
SUMMARY_TEXT=$(strip_bump_prefix "$SUMMARY")
else
SUMMARY_TEXT="Automated release v${VERSION}."
fi
COMMITS=$(commits_since "$PREV_TAG")
SECTION=$(changelog_section "$VERSION" "$DATE" "$SUMMARY_TEXT" "$COMMITS")
if [[ ! -f "$CHANGELOG" ]]; then
changelog_header > "$CHANGELOG"
printf '\n%s\n' "$SECTION" >> "$CHANGELOG"
else
UNRELEASED_MARKER='## [Unreleased]'
if ! grep -qF "$UNRELEASED_MARKER" "$CHANGELOG"; then
echo "CHANGELOG.md exists but is missing ${UNRELEASED_MARKER}" >&2
exit 1
fi
TMP=$(mktemp)
SECTION_FILE=$(mktemp)
printf '%s\n' "$SECTION" > "$SECTION_FILE"
awk -v section_file="$SECTION_FILE" '
/^## \[Unreleased\]/ {
print
print ""
while ((getline line < section_file) > 0) {
print line
}
close(section_file)
next
}
{ print }
' "$CHANGELOG" > "$TMP"
rm -f "$SECTION_FILE"
mv "$TMP" "$CHANGELOG"
fi
echo "Updated ${CHANGELOG} with version ${VERSION}." >&2
+6 -1
View File
@@ -1,6 +1,6 @@
"""Shared test configuration constants."""
from python_repositories.config import MinioConfig, RedisConfig
from python_repositories.config import MinioConfig, PostgresConfig, RedisConfig
TEST_REDIS_CONFIG = RedisConfig(uri="redis://localhost:6379")
TEST_MINIO_CONFIG = MinioConfig(
@@ -10,3 +10,8 @@ TEST_MINIO_CONFIG = MinioConfig(
bucket="test-bucket",
secure=False,
)
TEST_POSTGRES_CONFIG = PostgresConfig(
uri="postgresql://localhost/mydb",
table="test_items",
primary_key="id",
)
-135
View File
@@ -1,51 +1,9 @@
"""Integration tests configuration."""
import logging
from collections.abc import Generator
from typing import Any, cast
import pytest
import redis
import structlog
from minio import Minio
from testcontainers.core.container import DockerContainer
from testcontainers.core.waiting_utils import WaitStrategy, WaitStrategyTarget
from testcontainers.minio import MinioContainer
from python_repositories.config import MinioConfig, RedisConfig
REDIS_PORT = 6379
MINIO_ACCESS_KEY = "minioadmin"
MINIO_SECRET_KEY = "minioadmin"
MINIO_BUCKET = "test-bucket"
class _RedisPingWaitStrategy(WaitStrategy):
def __init__(self) -> None:
super().__init__()
self.with_transient_exceptions(redis.exceptions.ConnectionError)
def wait_until_ready(self, container: WaitStrategyTarget) -> None:
redis_container = cast("RedisTestContainer", container)
if not self._poll(lambda: redis_container.get_client().ping()):
raise redis.exceptions.ConnectionError("Could not connect to Redis")
class RedisTestContainer(DockerContainer):
"""Redis container using wait strategies instead of the deprecated decorator."""
def __init__(self, image: str, port: int = REDIS_PORT) -> None:
super().__init__(image, _wait_strategy=_RedisPingWaitStrategy())
self.port = port
self.with_exposed_ports(self.port)
def get_client(self, **kwargs: Any) -> redis.Redis:
return redis.Redis(
host=self.get_container_host_ip(),
port=self.get_exposed_port(self.port),
**kwargs,
)
@pytest.fixture(scope="session", autouse=True)
@@ -64,96 +22,3 @@ def configure_logging() -> None:
cache_logger_on_first_use=True,
)
logging.basicConfig(level=logging.ERROR)
@pytest.fixture(scope="session")
def redis_container() -> Generator[str, None, None]:
"""Set up a Redis container for testing and yield the Redis URI."""
container = RedisTestContainer(
image="redis/redis-stack:7.2.0-v0",
)
container.start()
redis_host = container.get_container_host_ip()
redis_port = container.get_exposed_port(REDIS_PORT)
redis_uri = f"redis://{redis_host}:{redis_port}"
yield redis_uri
container.stop()
@pytest.fixture(scope="session")
def minio_container() -> Generator[dict[str, str], None, None]:
"""Set up a Minio container for testing and yield connection settings."""
container = MinioContainer(
image="minio/minio:latest",
access_key=MINIO_ACCESS_KEY,
secret_key=MINIO_SECRET_KEY,
)
container.start()
minio_host = container.get_container_host_ip()
minio_port = container.get_exposed_port(9000)
minio_endpoint = f"{minio_host}:{minio_port}"
env_vars = {
"MINIO_ENDPOINT": minio_endpoint,
"MINIO_ACCESS_KEY": MINIO_ACCESS_KEY,
"MINIO_SECRET_KEY": MINIO_SECRET_KEY,
"MINIO_BUCKET": MINIO_BUCKET,
"MINIO_SECURE": "false",
}
yield env_vars
container.stop()
@pytest.fixture(scope="session")
def redis_config(redis_container: str) -> RedisConfig:
"""Provide RedisConfig built from the test container."""
return RedisConfig(uri=redis_container)
@pytest.fixture(scope="session")
def minio_config(minio_container: dict[str, str]) -> MinioConfig:
"""Provide MinioConfig built from the test container."""
return MinioConfig(
endpoint=minio_container["MINIO_ENDPOINT"],
access_key=minio_container["MINIO_ACCESS_KEY"],
secret_key=minio_container["MINIO_SECRET_KEY"],
bucket=minio_container["MINIO_BUCKET"],
secure=False,
)
@pytest.fixture(scope="session")
def raw_redis_client(redis_container: str) -> Generator[redis.Redis, None, None]:
"""Provide a raw Redis client connected to the test Redis container."""
client = redis.Redis.from_url(
url=redis_container,
socket_connect_timeout=10,
)
yield client
client.flushall()
client.close()
@pytest.fixture(scope="session")
def raw_minio_client(minio_container: dict[str, str]) -> Generator[Minio, None, None]:
"""Provide a raw Minio client connected to the test Minio container."""
client = Minio(
endpoint=minio_container["MINIO_ENDPOINT"],
access_key=minio_container["MINIO_ACCESS_KEY"],
secret_key=minio_container["MINIO_SECRET_KEY"],
secure=False,
)
bucket_name = minio_container["MINIO_BUCKET"]
if not client.bucket_exists(bucket_name):
client.make_bucket(bucket_name)
yield client
objects = client.list_objects(bucket_name, recursive=True)
for obj in objects:
client.remove_object(bucket_name, obj.object_name)
+80
View File
@@ -0,0 +1,80 @@
"""Examples integration test fixtures."""
from collections.abc import Generator
from minio import Minio
import pytest
import redis
from python_repositories.config import MinioConfig, PostgresConfig, RedisConfig
from tests.integration.minio._containers import (
minio_config_from_env,
minio_env,
raw_minio_client_from_env,
)
from tests.integration.postgres._containers import (
postgres_config_from_container,
postgres_uri,
raw_postgres_client_from_container,
)
from tests.integration.redis._containers import (
raw_redis_client_from_container,
redis_config_from_container,
redis_uri,
)
@pytest.fixture(scope="session")
def redis_container() -> Generator[str, None, None]:
"""Set up a Redis container for testing and yield the Redis URI."""
yield from redis_uri()
@pytest.fixture(scope="session")
def redis_config(redis_container: str) -> RedisConfig:
"""Provide RedisConfig built from the test container."""
return redis_config_from_container(redis_container)
@pytest.fixture(scope="session")
def raw_redis_client(redis_container: str) -> Generator[redis.Redis, None, None]:
"""Provide a raw Redis client connected to the test Redis container."""
yield from raw_redis_client_from_container(redis_container)
@pytest.fixture(scope="session")
def minio_container() -> Generator[dict[str, str], None, None]:
"""Set up a Minio container for testing and yield connection settings."""
yield from minio_env()
@pytest.fixture(scope="session")
def minio_config(minio_container: dict[str, str]) -> MinioConfig:
"""Provide MinioConfig built from the test container."""
return minio_config_from_env(minio_container)
@pytest.fixture(scope="session")
def raw_minio_client(minio_container: dict[str, str]) -> Generator[Minio, None, None]:
"""Provide a raw Minio client connected to the test Minio container."""
yield from raw_minio_client_from_env(minio_container)
@pytest.fixture(scope="session")
def postgres_container() -> Generator[str, None, None]:
"""Set up a Postgres container for testing and yield the Postgres URI."""
yield from postgres_uri()
@pytest.fixture(scope="session")
def postgres_config(postgres_container: str) -> PostgresConfig:
"""Provide PostgresConfig built from the test container."""
return postgres_config_from_container(postgres_container)
@pytest.fixture(scope="session")
def raw_postgres_client(
postgres_container: str,
) -> Generator[object, None, None]:
"""Provide a raw Postgres client connected to the test Postgres container."""
yield from raw_postgres_client_from_container(postgres_container)
@@ -5,15 +5,22 @@ from io import BytesIO
import os
import random
import pytest
from minio import Minio
import psycopg
import pytest
from python_repositories.examples.artifact_object_repository import (
ArtifactObjectRepository,
)
from python_repositories.examples.user_json_repository import UserJsonRepository
from python_repositories.examples.user_table_repository import UserTableRepository
from tests.integration.postgres._containers import TEST_TABLE
pytestmark = pytest.mark.integration
pytestmark = [
pytest.mark.integration,
pytest.mark.needs_redis,
pytest.mark.needs_minio,
]
@pytest.fixture(scope="module", autouse=True)
@@ -32,6 +39,24 @@ def set_example_env(
_ = os.environ.pop(key, default=None)
@pytest.fixture(scope="module", autouse=True)
def set_postgres_example_env(
postgres_container: str,
raw_postgres_client: psycopg.Connection,
) -> Generator[None, None, None]:
"""Set Postgres env vars for example repositories using from_env() defaults."""
env_vars = {
"POSTGRES_URI": postgres_container,
"POSTGRES_TABLE": TEST_TABLE,
"POSTGRES_PRIMARY_KEY": "id",
}
for key, value in env_vars.items():
os.environ[key] = value
yield
for key in env_vars:
_ = os.environ.pop(key, default=None)
@pytest.fixture(scope="module")
def user_data() -> Generator[dict[str, str], None, None]:
"""Provide sample user data for tests."""
@@ -73,3 +98,27 @@ def test_artifact_object_repository_store_and_get(
assert received is not None
artifact_data.seek(0)
assert received.read() == artifact_data.read()
@pytest.mark.needs_postgres
def test_user_table_repository_save_and_get(
user_data: dict[str, str],
) -> None:
"""Test that UserTableRepository can save and retrieve a user."""
with UserTableRepository() as repo:
repo.save_user("alice", user_data)
user = repo.get_user("alice")
assert user is not None
assert user["name"] == user_data["name"]
assert user["email"] == user_data["email"]
@pytest.mark.needs_postgres
def test_user_table_repository_delete(
user_data: dict[str, str],
) -> None:
"""Test that UserTableRepository can delete a user."""
with UserTableRepository() as repo:
repo.save_user("alice", user_data)
repo.delete_user("alice")
assert repo.get_user("alice") is None
+86
View File
@@ -0,0 +1,86 @@
"""MinIO container session helpers for integration tests."""
from collections.abc import Generator
from minio import Minio
from testcontainers.minio import MinioContainer
from python_repositories.config import MinioConfig
MINIO_ACCESS_KEY = "minioadmin"
MINIO_SECRET_KEY = "minioadmin"
MINIO_BUCKET = "test-bucket"
_minio_env: dict[str, str] | None = None
_minio_container: MinioContainer | None = None
_raw_minio_client: Minio | None = None
_minio_env_refs = 0
_raw_minio_client_refs = 0
def minio_env() -> Generator[dict[str, str], None, None]:
"""Yield session-scoped MinIO connection settings, starting the container once."""
global _minio_env, _minio_container, _minio_env_refs
if _minio_env is None:
_minio_container = MinioContainer(
image="minio/minio:latest",
access_key=MINIO_ACCESS_KEY,
secret_key=MINIO_SECRET_KEY,
)
_minio_container.start()
minio_host = _minio_container.get_container_host_ip()
minio_port = _minio_container.get_exposed_port(9000)
minio_endpoint = f"{minio_host}:{minio_port}"
_minio_env = {
"MINIO_ENDPOINT": minio_endpoint,
"MINIO_ACCESS_KEY": MINIO_ACCESS_KEY,
"MINIO_SECRET_KEY": MINIO_SECRET_KEY,
"MINIO_BUCKET": MINIO_BUCKET,
"MINIO_SECURE": "false",
}
_minio_env_refs += 1
yield _minio_env
_minio_env_refs -= 1
if _minio_env_refs == 0 and _minio_container is not None:
_minio_container.stop()
_minio_container = None
_minio_env = None
def minio_config_from_env(env: dict[str, str]) -> MinioConfig:
"""Build MinioConfig from container environment settings."""
return MinioConfig(
endpoint=env["MINIO_ENDPOINT"],
access_key=env["MINIO_ACCESS_KEY"],
secret_key=env["MINIO_SECRET_KEY"],
bucket=env["MINIO_BUCKET"],
secure=False,
)
def raw_minio_client_from_env(env: dict[str, str]) -> Generator[Minio, None, None]:
"""Yield a session-scoped raw MinIO client, reusing one client per session."""
global _raw_minio_client, _raw_minio_client_refs
if _raw_minio_client is None:
_raw_minio_client = Minio(
endpoint=env["MINIO_ENDPOINT"],
access_key=env["MINIO_ACCESS_KEY"],
secret_key=env["MINIO_SECRET_KEY"],
secure=False,
)
bucket_name = env["MINIO_BUCKET"]
if not _raw_minio_client.bucket_exists(bucket_name):
_raw_minio_client.make_bucket(bucket_name)
_raw_minio_client_refs += 1
yield _raw_minio_client
_raw_minio_client_refs -= 1
if _raw_minio_client_refs == 0 and _raw_minio_client is not None:
bucket_name = env["MINIO_BUCKET"]
objects = _raw_minio_client.list_objects(bucket_name, recursive=True)
for obj in objects:
_raw_minio_client.remove_object(bucket_name, obj.object_name)
_raw_minio_client = None
+31
View File
@@ -0,0 +1,31 @@
"""MinIO integration test fixtures."""
from collections.abc import Generator
from minio import Minio
import pytest
from python_repositories.config import MinioConfig
from tests.integration.minio._containers import (
minio_config_from_env,
minio_env,
raw_minio_client_from_env,
)
@pytest.fixture(scope="session")
def minio_container() -> Generator[dict[str, str], None, None]:
"""Set up a Minio container for testing and yield connection settings."""
yield from minio_env()
@pytest.fixture(scope="session")
def minio_config(minio_container: dict[str, str]) -> MinioConfig:
"""Provide MinioConfig built from the test container."""
return minio_config_from_env(minio_container)
@pytest.fixture(scope="session")
def raw_minio_client(minio_container: dict[str, str]) -> Generator[Minio, None, None]:
"""Provide a raw Minio client connected to the test Minio container."""
yield from raw_minio_client_from_env(minio_container)
@@ -2,20 +2,20 @@
from collections.abc import Generator
from dataclasses import replace
from io import BytesIO
import logging
import random
from io import BytesIO
from unittest.mock import MagicMock
import pytest
from minio import Minio, S3Error
import pytest
from urllib3.response import BaseHTTPResponse
from python_repositories.adapters.minio_adapter import MinioAdapter
from python_repositories.config import MinioConfig
from tests.conftest import TEST_MINIO_CONFIG
pytestmark = pytest.mark.integration
pytestmark = [pytest.mark.integration, pytest.mark.needs_minio]
def same_data(
@@ -328,6 +328,44 @@ def test_should_raise_value_error_on_invalid_put_data(
minio_adapter.put(object_name, invalid) # type: ignore
def test_should_put_and_get_empty_bytesio(
minio_adapter: MinioAdapter,
minio_config: MinioConfig,
) -> None:
"""Test that the MinioAdapter can put and get a zero-byte object."""
object_name = "empty_object"
empty_data = BytesIO()
assert minio_adapter.get(object_name) is None
minio_adapter.put(object_name, empty_data)
received_data = minio_adapter.get(object_name)
assert received_data is not None
assert same_data(empty_data, received_data)
minio_adapter._client.remove_object(minio_config.bucket, object_name) # type: ignore[union-attr]
def test_should_distinguish_missing_object_from_empty_object(
minio_adapter: MinioAdapter,
minio_config: MinioConfig,
) -> None:
"""Test that missing objects and zero-byte objects are distinguishable."""
object_name = "empty_object"
minio_adapter.put(object_name, BytesIO())
assert minio_adapter.get("other_object") is None
minio_adapter.delete(object_name)
assert minio_adapter.get(object_name) is None
def test_should_list_zero_byte_object(
minio_adapter: MinioAdapter,
minio_config: MinioConfig,
) -> None:
"""Test that a zero-byte object appears in object listings."""
object_name = "empty_object"
minio_adapter.put(object_name, BytesIO())
assert object_name in minio_adapter.list_objects()
minio_adapter._client.remove_object(minio_config.bucket, object_name) # type: ignore[union-attr]
def test_should_raise_value_error_on_invalid_put_content_type(
data: BytesIO,
minio_adapter: MinioAdapter,
+76
View File
@@ -0,0 +1,76 @@
"""Postgres container session helpers for integration tests."""
from collections.abc import Generator
import psycopg
from psycopg.rows import dict_row
from testcontainers.postgres import PostgresContainer
from python_repositories.config import PostgresConfig
POSTGRES_IMAGE = "postgres:16"
TEST_TABLE = "test_items"
_postgres_uri: str | None = None
_postgres_container: PostgresContainer | None = None
_raw_postgres_client: psycopg.Connection | None = None
_postgres_uri_refs = 0
_raw_postgres_client_refs = 0
def postgres_uri() -> Generator[str, None, None]:
"""Yield a session-scoped Postgres URI, starting the container once."""
global _postgres_uri, _postgres_container, _postgres_uri_refs
if _postgres_uri is None:
_postgres_container = PostgresContainer(POSTGRES_IMAGE, driver=None)
_postgres_container.start()
_postgres_uri = _postgres_container.get_connection_url()
_postgres_uri_refs += 1
yield _postgres_uri
_postgres_uri_refs -= 1
if _postgres_uri_refs == 0 and _postgres_container is not None:
_postgres_container.stop()
_postgres_container = None
_postgres_uri = None
def postgres_config_from_container(uri: str) -> PostgresConfig:
"""Build PostgresConfig from a container URI."""
return PostgresConfig(uri=uri, table=TEST_TABLE, primary_key="id")
def raw_postgres_client_from_container(
uri: str,
) -> Generator[psycopg.Connection, None, None]:
"""Yield a session-scoped raw Postgres client, reusing one client per session."""
global _raw_postgres_client, _raw_postgres_client_refs
if _raw_postgres_client is None:
_raw_postgres_client = psycopg.connect(
uri,
row_factory=dict_row, # type: ignore[arg-type]
autocommit=True,
)
with _raw_postgres_client.cursor() as cursor:
cursor.execute(
f"""
CREATE TABLE IF NOT EXISTS {TEST_TABLE} (
id TEXT PRIMARY KEY,
name TEXT NOT NULL,
value INTEGER,
email TEXT
)
"""
)
_raw_postgres_client_refs += 1
yield _raw_postgres_client
_raw_postgres_client_refs -= 1
if _raw_postgres_client_refs == 0 and _raw_postgres_client is not None:
with _raw_postgres_client.cursor() as cursor:
cursor.execute(f"TRUNCATE TABLE {TEST_TABLE}")
_raw_postgres_client.close()
_raw_postgres_client = None
+33
View File
@@ -0,0 +1,33 @@
"""Postgres integration test fixtures."""
from collections.abc import Generator
import psycopg
import pytest
from python_repositories.config import PostgresConfig
from tests.integration.postgres._containers import (
postgres_config_from_container,
postgres_uri,
raw_postgres_client_from_container,
)
@pytest.fixture(scope="session")
def postgres_container() -> Generator[str, None, None]:
"""Set up a Postgres container for testing and yield the Postgres URI."""
yield from postgres_uri()
@pytest.fixture(scope="session")
def postgres_config(postgres_container: str) -> PostgresConfig:
"""Provide PostgresConfig built from the test container."""
return postgres_config_from_container(postgres_container)
@pytest.fixture(scope="session")
def raw_postgres_client(
postgres_container: str,
) -> Generator[psycopg.Connection, None, None]:
"""Provide a raw Postgres client connected to the test container."""
yield from raw_postgres_client_from_container(postgres_container)
@@ -0,0 +1,318 @@
"""Integration tests for the PostgresAdapter."""
from collections.abc import Generator
import logging
import psycopg
import pytest
from python_repositories.adapters.postgres_adapter import PostgresAdapter
from python_repositories.config import PostgresConfig
from tests.integration.postgres._containers import TEST_TABLE
pytestmark = [pytest.mark.integration, pytest.mark.needs_postgres]
@pytest.fixture(scope="module")
def row() -> Generator[dict[str, object], None, None]:
"""Provide a sample row for tests."""
yield {"id": "test-id", "name": "Alice", "value": 42}
@pytest.fixture(scope="function")
def row_in_postgres(
raw_postgres_client: psycopg.Connection,
row: dict[str, object],
) -> Generator[tuple[str, dict[str, object]], None, None]:
"""Fixture to set up a known row in Postgres before each test."""
with raw_postgres_client.cursor() as cursor:
cursor.execute(
f"INSERT INTO {TEST_TABLE} (id, name, value) VALUES (%s, %s, %s)",
(row["id"], row["name"], row["value"]),
)
yield str(row["id"]), row
with raw_postgres_client.cursor() as cursor:
cursor.execute(f"DELETE FROM {TEST_TABLE} WHERE id = %s", (row["id"],))
@pytest.fixture(scope="module")
def postgres_adapter(
postgres_config: PostgresConfig,
raw_postgres_client: psycopg.Connection,
) -> Generator[PostgresAdapter, None, None]:
"""Fixture to provide a connected PostgresAdapter instance."""
_ = raw_postgres_client # ensure test table exists before connect probe
adapter = PostgresAdapter(config=postgres_config)
adapter.connect()
yield adapter
adapter.disconnect()
@pytest.fixture(scope="function", autouse=True)
def clear_postgres(raw_postgres_client: psycopg.Connection) -> None:
"""Fixture to clear all rows before each test."""
with raw_postgres_client.cursor() as cursor:
cursor.execute(f"TRUNCATE TABLE {TEST_TABLE}")
def test_should_log_info_when_already_connected(
postgres_adapter: PostgresAdapter,
caplog: pytest.LogCaptureFixture,
) -> None:
"""Test that PostgresAdapter logs info when connect is called while connected."""
with caplog.at_level(logging.INFO):
postgres_adapter.connect()
assert "Already connected" in caplog.text
assert "Postgres" in caplog.text
def test_should_raise_connection_error_when_unable_to_connect() -> None:
"""Test that PostgresAdapter raises ConnectionError when unable to connect."""
adapter = PostgresAdapter(
config=PostgresConfig(
uri="postgresql://invalid:5432/nodb",
table=TEST_TABLE,
)
)
with pytest.raises(ConnectionError):
adapter.connect()
assert adapter._client is None
assert not adapter.is_connected()
def test_should_raise_connection_error_when_table_missing(
postgres_config: PostgresConfig,
) -> None:
"""Test that PostgresAdapter raises ConnectionError when the table is missing."""
config = PostgresConfig(
uri=postgres_config.uri,
table="missing_table",
primary_key="id",
)
adapter = PostgresAdapter(config=config)
with pytest.raises(ConnectionError, match="does not exist"):
adapter.connect()
def test_should_log_error_on_exception_during_exit(
postgres_config: PostgresConfig,
caplog: pytest.LogCaptureFixture,
) -> None:
"""Test that PostgresAdapter logs an error when an exception occurs during exit."""
try:
with PostgresAdapter(config=postgres_config) as adapter:
assert adapter.is_connected()
raise ValueError("Simulated error")
except ValueError:
pass
assert "Error while exiting context" in caplog.text
def test_should_have_context_manager(postgres_config: PostgresConfig) -> None:
"""Test that PostgresAdapter can be used as a context manager."""
with PostgresAdapter(config=postgres_config) as adapter:
assert adapter._client is not None
assert adapter._client is None
def test_should_fetch_one(
row_in_postgres: tuple[str, dict[str, object]],
postgres_adapter: PostgresAdapter,
) -> None:
"""Test that PostgresAdapter can fetch a row by primary key."""
pk, row = row_in_postgres
fetched = postgres_adapter.fetch_one(pk)
assert fetched is not None
assert fetched["id"] == row["id"]
assert fetched["name"] == row["name"]
assert fetched["value"] == row["value"]
def test_should_fetch_none_for_missing_row(
postgres_adapter: PostgresAdapter,
) -> None:
"""Test that fetching a non-existent row returns None."""
assert postgres_adapter.fetch_one("missing-id") is None
def test_should_raise_value_error_on_invalid_fetch_one_pk(
postgres_adapter: PostgresAdapter,
) -> None:
"""Test that PostgresAdapter raises ValueError for an invalid primary key."""
with pytest.raises(ValueError):
postgres_adapter.fetch_one(None)
def test_should_raise_connection_error_on_fetch_one_when_not_connected(
postgres_config: PostgresConfig,
) -> None:
"""Test that fetch_one raises ConnectionError when not connected."""
adapter = PostgresAdapter(config=postgres_config)
with pytest.raises(ConnectionError):
adapter.fetch_one("some-id")
def test_should_upsert_row(
row: dict[str, object],
postgres_adapter: PostgresAdapter,
) -> None:
"""Test that PostgresAdapter can insert a row."""
assert postgres_adapter.fetch_one(row["id"]) is None
postgres_adapter.upsert(row)
fetched = postgres_adapter.fetch_one(row["id"])
assert fetched is not None
assert fetched["name"] == row["name"]
def test_should_update_row(
row_in_postgres: tuple[str, dict[str, object]],
postgres_adapter: PostgresAdapter,
) -> None:
"""Test that PostgresAdapter can update an existing row."""
pk, _ = row_in_postgres
new_row = {"id": pk, "name": "Bob", "value": 99}
postgres_adapter.upsert(new_row)
fetched = postgres_adapter.fetch_one(pk)
assert fetched is not None
assert fetched["name"] == "Bob"
assert fetched["value"] == 99
def test_should_raise_value_error_on_invalid_upsert_row(
postgres_adapter: PostgresAdapter,
) -> None:
"""Test that PostgresAdapter raises ValueError for invalid upsert data."""
with pytest.raises(ValueError):
postgres_adapter.upsert({"name": "Alice"}) # missing primary key
def test_should_raise_value_error_on_non_dict_upsert_row(
postgres_adapter: PostgresAdapter,
) -> None:
"""Test that PostgresAdapter raises ValueError when upsert row is not a dict."""
with pytest.raises(ValueError):
postgres_adapter.upsert("not-a-dict") # type: ignore[arg-type]
def test_should_raise_connection_error_on_upsert_when_not_connected(
postgres_config: PostgresConfig,
row: dict[str, object],
) -> None:
"""Test that upsert raises ConnectionError when not connected."""
adapter = PostgresAdapter(config=postgres_config)
with pytest.raises(ConnectionError):
adapter.upsert(row)
def test_should_fetch_all_rows(
postgres_adapter: PostgresAdapter,
) -> None:
"""Test that PostgresAdapter can fetch all rows."""
postgres_adapter.upsert({"id": "a", "name": "Alice", "value": 1})
postgres_adapter.upsert({"id": "b", "name": "Bob", "value": 2})
rows = postgres_adapter.fetch_all()
assert len(rows) == 2
ids = {row["id"] for row in rows}
assert ids == {"a", "b"}
def test_should_fetch_all_with_limit(
postgres_adapter: PostgresAdapter,
) -> None:
"""Test that PostgresAdapter respects fetch_all limit."""
postgres_adapter.upsert({"id": "a", "name": "Alice", "value": 1})
postgres_adapter.upsert({"id": "b", "name": "Bob", "value": 2})
rows = postgres_adapter.fetch_all(limit=1)
assert len(rows) == 1
def test_should_return_empty_list_for_empty_table(
postgres_adapter: PostgresAdapter,
) -> None:
"""Test that fetch_all returns an empty list for an empty table."""
assert postgres_adapter.fetch_all() == []
def test_should_raise_value_error_on_invalid_fetch_all_limit(
postgres_adapter: PostgresAdapter,
) -> None:
"""Test that PostgresAdapter raises ValueError for an invalid limit."""
with pytest.raises(ValueError):
postgres_adapter.fetch_all(limit=-1)
def test_should_raise_connection_error_on_fetch_all_when_not_connected(
postgres_config: PostgresConfig,
) -> None:
"""Test that fetch_all raises ConnectionError when not connected."""
adapter = PostgresAdapter(config=postgres_config)
with pytest.raises(ConnectionError):
adapter.fetch_all()
def test_should_delete_row(
row_in_postgres: tuple[str, dict[str, object]],
postgres_adapter: PostgresAdapter,
) -> None:
"""Test that deleting a row removes it from the table."""
pk, _ = row_in_postgres
assert postgres_adapter.fetch_one(pk) is not None
postgres_adapter.delete(pk)
assert postgres_adapter.fetch_one(pk) is None
def test_should_delete_idempotently(
postgres_adapter: PostgresAdapter,
) -> None:
"""Test that deleting a missing row does not raise."""
postgres_adapter.delete("missing-id")
def test_should_raise_value_error_on_invalid_delete_pk(
postgres_adapter: PostgresAdapter,
) -> None:
"""Test that PostgresAdapter raises ValueError for an invalid delete pk."""
with pytest.raises(ValueError):
postgres_adapter.delete(None)
def test_should_raise_connection_error_on_delete_when_not_connected(
postgres_config: PostgresConfig,
) -> None:
"""Test that delete raises ConnectionError when not connected."""
adapter = PostgresAdapter(config=postgres_config)
with pytest.raises(ConnectionError):
adapter.delete("some-id")
def test_should_execute_sql(
row_in_postgres: tuple[str, dict[str, object]],
postgres_adapter: PostgresAdapter,
) -> None:
"""Test that PostgresAdapter can execute read SQL."""
pk, _ = row_in_postgres
rows = postgres_adapter.execute(
f"SELECT * FROM {TEST_TABLE} WHERE id = %s",
(pk,),
)
assert len(rows) == 1
assert rows[0]["id"] == pk
def test_should_raise_value_error_on_invalid_execute_sql(
postgres_adapter: PostgresAdapter,
) -> None:
"""Test that PostgresAdapter raises ValueError for invalid SQL."""
with pytest.raises(ValueError):
postgres_adapter.execute("")
def test_should_raise_connection_error_on_execute_when_not_connected(
postgres_config: PostgresConfig,
) -> None:
"""Test that execute raises ConnectionError when not connected."""
adapter = PostgresAdapter(config=postgres_config)
with pytest.raises(ConnectionError):
adapter.execute("SELECT 1")
+90
View File
@@ -0,0 +1,90 @@
"""Redis container session helpers for integration tests."""
from collections.abc import Generator
from typing import Any, cast
import redis
from testcontainers.core.container import DockerContainer
from testcontainers.core.waiting_utils import WaitStrategy, WaitStrategyTarget
from python_repositories.config import RedisConfig
REDIS_PORT = 6379
class _RedisPingWaitStrategy(WaitStrategy):
def __init__(self) -> None:
super().__init__()
self.with_transient_exceptions(redis.exceptions.ConnectionError)
def wait_until_ready(self, container: WaitStrategyTarget) -> None:
redis_container = cast("RedisTestContainer", container)
if not self._poll(lambda: redis_container.get_client().ping()):
raise redis.exceptions.ConnectionError("Could not connect to Redis")
class RedisTestContainer(DockerContainer):
"""Redis container using wait strategies instead of the deprecated decorator."""
def __init__(self, image: str, port: int = REDIS_PORT) -> None:
super().__init__(image, _wait_strategy=_RedisPingWaitStrategy())
self.port = port
self.with_exposed_ports(self.port)
def get_client(self, **kwargs: Any) -> redis.Redis:
return redis.Redis(
host=self.get_container_host_ip(),
port=self.get_exposed_port(self.port),
**kwargs,
)
_redis_uri: str | None = None
_redis_container: RedisTestContainer | None = None
_raw_redis_client: redis.Redis | None = None
_redis_uri_refs = 0
_raw_redis_client_refs = 0
def redis_uri() -> Generator[str, None, None]:
"""Yield a session-scoped Redis URI, starting the container once."""
global _redis_uri, _redis_container, _redis_uri_refs
if _redis_uri is None:
_redis_container = RedisTestContainer(image="redis/redis-stack:7.2.0-v0")
_redis_container.start()
redis_host = _redis_container.get_container_host_ip()
redis_port = _redis_container.get_exposed_port(REDIS_PORT)
_redis_uri = f"redis://{redis_host}:{redis_port}"
_redis_uri_refs += 1
yield _redis_uri
_redis_uri_refs -= 1
if _redis_uri_refs == 0 and _redis_container is not None:
_redis_container.stop()
_redis_container = None
_redis_uri = None
def redis_config_from_container(uri: str) -> RedisConfig:
"""Build RedisConfig from a container URI."""
return RedisConfig(uri=uri)
def raw_redis_client_from_container(uri: str) -> Generator[redis.Redis, None, None]:
"""Yield a session-scoped raw Redis client, reusing one client per session."""
global _raw_redis_client, _raw_redis_client_refs
if _raw_redis_client is None:
_raw_redis_client = redis.Redis.from_url(
url=uri,
socket_connect_timeout=10,
)
_raw_redis_client_refs += 1
yield _raw_redis_client
_raw_redis_client_refs -= 1
if _raw_redis_client_refs == 0 and _raw_redis_client is not None:
_raw_redis_client.flushall()
_raw_redis_client.close()
_raw_redis_client = None
+31
View File
@@ -0,0 +1,31 @@
"""Redis integration test fixtures."""
from collections.abc import Generator
import pytest
import redis
from python_repositories.config import RedisConfig
from tests.integration.redis._containers import (
raw_redis_client_from_container,
redis_config_from_container,
redis_uri,
)
@pytest.fixture(scope="session")
def redis_container() -> Generator[str, None, None]:
"""Set up a Redis container for testing and yield the Redis URI."""
yield from redis_uri()
@pytest.fixture(scope="session")
def redis_config(redis_container: str) -> RedisConfig:
"""Provide RedisConfig built from the test container."""
return redis_config_from_container(redis_container)
@pytest.fixture(scope="session")
def raw_redis_client(redis_container: str) -> Generator[redis.Redis, None, None]:
"""Provide a raw Redis client connected to the test Redis container."""
yield from raw_redis_client_from_container(redis_container)
@@ -10,7 +10,7 @@ from redis.commands.json.path import Path as RedisPath
from python_repositories.adapters.redis_adapter import RedisAdapter
from python_repositories.config import RedisConfig
pytestmark = pytest.mark.integration
pytestmark = [pytest.mark.integration, pytest.mark.needs_redis]
@pytest.fixture(scope="module")
@@ -190,12 +190,45 @@ def test_should_raise_value_error_on_invalid_set_data(
) -> None:
"""Test that the RedisAdapter raises ValueError when setting with invalid data."""
key = "test_key"
invalid_data = ["", 123, None, [], {}]
invalid_data = ["", 123, None, []]
for data in invalid_data:
with pytest.raises(ValueError):
redis_adapter.set(key, data) # type: ignore
def test_should_set_and_get_empty_dict(
redis_adapter: RedisAdapter,
) -> None:
"""Test that the RedisAdapter can set and get an empty dict."""
key = "empty_key"
assert redis_adapter.get(key) is None
redis_adapter.set(key, {})
value = redis_adapter.get(key)
assert value is not None
assert value == {}
def test_should_distinguish_missing_key_from_empty_dict(
redis_adapter: RedisAdapter,
) -> None:
"""Test that missing keys and empty dicts are distinguishable."""
key = "empty_key"
redis_adapter.set(key, {})
assert redis_adapter.get("other_key") is None
redis_adapter.delete(key)
assert redis_adapter.get(key) is None
def test_should_list_empty_dict_key(
redis_adapter: RedisAdapter,
) -> None:
"""Test that a key with an empty dict appears in key listings."""
key = "empty_key"
redis_adapter.set(key, {})
assert key in redis_adapter.list_keys(key)
assert key in list(redis_adapter.scan_keys(key))
def test_should_raise_connection_error_on_set_when_not_connected(
redis_config: RedisConfig,
data: dict[str, str],
+49 -4
View File
@@ -3,12 +3,12 @@
from __future__ import annotations
import builtins
from collections.abc import Callable, Mapping, Sequence
import importlib
import os
from pathlib import Path
import subprocess
import sys
from collections.abc import Callable, Mapping, Sequence
from pathlib import Path
from types import ModuleType
from unittest.mock import patch
@@ -45,6 +45,9 @@ from python_repositories import JsonRepositoryInterface
assert JsonRepositoryInterface is not None
assert "python_repositories.adapters.redis_adapter" not in sys.modules
assert "python_repositories.adapters.minio_adapter" not in sys.modules
assert "python_repositories.adapters.postgres_adapter" not in sys.modules
assert "python_repositories.adapters.memory_queue_adapter" not in sys.modules
assert "python_repositories.adapters.file_backed_queue_adapter" not in sys.modules
"""
result = subprocess.run(
[sys.executable, "-c", script],
@@ -58,10 +61,19 @@ assert "python_repositories.adapters.minio_adapter" not in sys.modules
def test_lazy_adapter_load_succeeds_when_extra_present() -> None:
"""Adapters load when their optional dependencies are installed."""
from python_repositories import MinioAdapter, RedisAdapter
from python_repositories import (
FileBackedQueueAdapter,
MemoryQueueAdapter,
MinioAdapter,
PostgresAdapter,
RedisAdapter,
)
assert RedisAdapter.__name__ == "RedisAdapter"
assert MinioAdapter.__name__ == "MinioAdapter"
assert PostgresAdapter.__name__ == "PostgresAdapter"
assert MemoryQueueAdapter.__name__ == "MemoryQueueAdapter"
assert FileBackedQueueAdapter.__name__ == "FileBackedQueueAdapter"
def test_redis_adapter_import_error_without_extra() -> None:
@@ -86,6 +98,17 @@ def test_minio_adapter_import_error_without_extra() -> None:
importlib.reload(minio_adapter_module)
def test_postgres_adapter_import_error_without_extra() -> None:
"""Missing postgres extra raises ImportError with install hint."""
import python_repositories.adapters.postgres_adapter as postgres_adapter_module
with patch.object(builtins, "__import__", new=_block_backend_import("psycopg")):
with pytest.raises(ImportError, match=r"python-repositories\[postgres\]"):
importlib.reload(postgres_adapter_module)
importlib.reload(postgres_adapter_module)
def test_top_level_lazy_import_propagates_redis_import_error() -> None:
"""Top-level RedisAdapter access surfaces adapter import errors."""
with patch(
@@ -112,6 +135,19 @@ def test_top_level_lazy_import_propagates_minio_import_error() -> None:
_ = python_repositories.MinioAdapter
def test_top_level_lazy_import_propagates_postgres_import_error() -> None:
"""Top-level PostgresAdapter access surfaces adapter import errors."""
with patch(
"importlib.import_module",
side_effect=ImportError(
"Postgres support requires the postgres extra. "
"Install with: pip install python-repositories[postgres]"
),
):
with pytest.raises(ImportError, match=r"python-repositories\[postgres\]"):
_ = python_repositories.PostgresAdapter
def test_adapters_subpackage_lazy_import_succeeds() -> None:
"""Adapter subpackage imports delegate to the same lazy loader."""
from python_repositories.adapters import RedisAdapter
@@ -123,7 +159,13 @@ def test_adapters_dir_exposes_lazy_exports() -> None:
"""dir(adapters) includes lazy adapter names for tab completion."""
import python_repositories.adapters as adapters
assert {"RedisAdapter", "MinioAdapter"}.issubset(set(dir(adapters)))
assert {
"RedisAdapter",
"MinioAdapter",
"PostgresAdapter",
"MemoryQueueAdapter",
"FileBackedQueueAdapter",
}.issubset(set(dir(adapters)))
def test_adapters_getattr_raises_for_unknown() -> None:
@@ -138,3 +180,6 @@ def test_top_level_dir_exposes_lazy_exports() -> None:
"""dir(python_repositories) includes lazy adapter names for tab completion."""
assert "RedisAdapter" in dir(python_repositories)
assert "MinioAdapter" in dir(python_repositories)
assert "PostgresAdapter" in dir(python_repositories)
assert "MemoryQueueAdapter" in dir(python_repositories)
assert "FileBackedQueueAdapter" in dir(python_repositories)
+15 -2
View File
@@ -4,13 +4,15 @@ from __future__ import annotations
from unittest.mock import MagicMock
from minio import Minio
import psycopg
import pytest
import redis
from minio import Minio
from python_repositories.adapters.minio_adapter import MinioAdapter
from python_repositories.adapters.postgres_adapter import PostgresAdapter
from python_repositories.adapters.redis_adapter import RedisAdapter
from tests.conftest import TEST_MINIO_CONFIG, TEST_REDIS_CONFIG
from tests.conftest import TEST_MINIO_CONFIG, TEST_POSTGRES_CONFIG, TEST_REDIS_CONFIG
@pytest.fixture
@@ -25,3 +27,14 @@ def minio_adapter() -> MinioAdapter:
"""Provide a MinioAdapter with an injected mock client."""
mock_client = MagicMock(spec=Minio)
return MinioAdapter(config=TEST_MINIO_CONFIG, client=mock_client)
@pytest.fixture
def postgres_adapter() -> PostgresAdapter:
"""Provide a PostgresAdapter with an injected mock client."""
mock_client = MagicMock(spec=psycopg.Connection)
mock_cursor = MagicMock()
mock_cursor.__enter__ = MagicMock(return_value=mock_cursor)
mock_cursor.__exit__ = MagicMock(return_value=False)
mock_client.cursor.return_value = mock_cursor
return PostgresAdapter(config=TEST_POSTGRES_CONFIG, client=mock_client)
+74 -1
View File
@@ -5,11 +5,13 @@ from __future__ import annotations
from typing import cast
from unittest.mock import MagicMock, patch
import psycopg
import redis
from python_repositories.adapters.minio_adapter import MinioAdapter
from python_repositories.adapters.postgres_adapter import PostgresAdapter
from python_repositories.adapters.redis_adapter import RedisAdapter
from tests.conftest import TEST_MINIO_CONFIG, TEST_REDIS_CONFIG
from tests.conftest import TEST_MINIO_CONFIG, TEST_POSTGRES_CONFIG, TEST_REDIS_CONFIG
class TestRedisConnectionHealth:
@@ -161,3 +163,74 @@ class TestMinioConnectionHealth:
assert reinjected.is_connected()
assert mock_client.bucket_exists.call_count == 2
class TestPostgresConnectionHealth:
def test_not_connected_when_no_client(self) -> None:
adapter = PostgresAdapter(config=TEST_POSTGRES_CONFIG)
assert not adapter.is_connected()
def test_connected_when_probe_succeeds(
self, postgres_adapter: PostgresAdapter
) -> None:
assert postgres_adapter.is_connected()
cast(MagicMock, postgres_adapter._client).cursor.assert_called()
def test_stale_connection_when_probe_fails(
self, postgres_adapter: PostgresAdapter
) -> None:
mock_cursor = cast(MagicMock, postgres_adapter._client).cursor.return_value
mock_cursor.execute.side_effect = psycopg.OperationalError("connection lost")
assert not postgres_adapter.is_connected()
def test_cache_hit_avoids_second_probe(
self, postgres_adapter: PostgresAdapter
) -> None:
mock_client = cast(MagicMock, postgres_adapter._client)
with patch(
"python_repositories.adapters.connection_aware_adapter.time.monotonic",
return_value=100.0,
):
assert postgres_adapter.is_connected()
assert postgres_adapter.is_connected()
assert mock_client.cursor.call_count == 1
def test_cache_miss_runs_probe_again(
self, postgres_adapter: PostgresAdapter
) -> None:
mock_client = cast(MagicMock, postgres_adapter._client)
with patch(
"python_repositories.adapters.connection_aware_adapter.time.monotonic",
side_effect=[100.0, 102.0],
):
assert postgres_adapter.is_connected()
assert postgres_adapter.is_connected()
assert mock_client.cursor.call_count == 2
def test_disconnect_clears_cache(self, postgres_adapter: PostgresAdapter) -> None:
mock_client = cast(MagicMock, postgres_adapter._client)
with patch(
"python_repositories.adapters.connection_aware_adapter.time.monotonic",
return_value=100.0,
):
assert postgres_adapter.is_connected()
postgres_adapter.disconnect()
reinjected = PostgresAdapter(
config=postgres_adapter._config,
client=mock_client,
)
with patch(
"python_repositories.adapters.connection_aware_adapter.time.monotonic",
return_value=100.0,
):
assert reinjected.is_connected()
assert mock_client.cursor.call_count == 2
@@ -1,11 +1,30 @@
"""Unit tests for ConnectionAwareInterface."""
import pytest
from python_repositories.interfaces.connection_aware_interface import (
ConnectionAwareInterface,
)
class FakeConnection:
"""Plain class that satisfies ConnectionAwareInterface without inheritance."""
def connect(self) -> None:
pass
def disconnect(self) -> None:
pass
def is_connected(self) -> bool:
return True
def accepts_connection_aware(connection: ConnectionAwareInterface) -> None:
"""Type-checking hook for ConnectionAwareInterface structural subtyping."""
connection.is_connected()
def test_instantiation_fails_when_connect_not_implemented() -> None:
"""Test that instantiation fails if connect is not implemented."""
@@ -52,3 +71,10 @@ def test_instantiation_fails_when_is_connected_not_implemented() -> None:
with pytest.raises(TypeError):
_ = Incomplete() # type: ignore
def test_structural_subtyping() -> None:
"""Test that a plain class satisfies ConnectionAwareInterface structurally."""
connection: ConnectionAwareInterface = FakeConnection()
accepts_connection_aware(connection)
assert isinstance(connection, ConnectionAwareInterface)
@@ -3,9 +3,28 @@
from __future__ import annotations
import pytest
from python_repositories.interfaces.context_aware_interface import ContextAwareInterface
class FakeContextManager:
"""Plain class that satisfies ContextAwareInterface without inheritance."""
def __enter__(self) -> FakeContextManager:
return self
def __exit__(
self, exc_type: type | None, exc_val: object | None, exc_tb: object | None
) -> None:
pass
def accepts_context_aware(context: ContextAwareInterface) -> None:
"""Type-checking hook for ContextAwareInterface structural subtyping."""
with context:
pass
def test_instantiation_fails_when_enter_not_implemented() -> None:
"""Test that instantiation fails if __enter__ is not implemented."""
@@ -32,3 +51,10 @@ def test_instantiation_fails_when_exit_not_implemented() -> None:
with pytest.raises(TypeError):
_ = Incomplete() # type: ignore
def test_structural_subtyping() -> None:
"""Test that a plain class satisfies ContextAwareInterface structurally."""
context: ContextAwareInterface = FakeContextManager()
accepts_context_aware(context)
assert isinstance(context, ContextAwareInterface)
@@ -0,0 +1,239 @@
"""Unit tests for FileBackedQueueAdapter."""
from __future__ import annotations
from datetime import UTC, datetime, timedelta
from pathlib import Path
import pytest
from python_repositories.adapters.file_backed_queue_adapter import (
FileBackedQueueAdapter,
)
from python_repositories.adapters.memory_queue_adapter import MemoryQueueAdapter
from python_repositories.config.file_queue_config import FileQueueConfig
from python_repositories.interfaces.queue_repository_interface import (
QueueRepositoryInterface,
)
def _config(path: Path, **kwargs: object) -> FileQueueConfig:
return FileQueueConfig(path=path, **kwargs) # type: ignore[arg-type]
def test_implements_interface() -> None:
assert issubclass(FileBackedQueueAdapter, QueueRepositoryInterface)
def test_ops_require_connect(tmp_path: Path) -> None:
queue = FileBackedQueueAdapter(config=_config(tmp_path / "q.jsonl"))
with pytest.raises(RuntimeError, match="not connected"):
queue.enqueue([{"id": 1}])
with pytest.raises(RuntimeError, match="not connected"):
queue.dequeue_batch()
with pytest.raises(RuntimeError, match="not connected"):
queue.size()
with pytest.raises(RuntimeError, match="not connected"):
queue.evict_older_than(datetime.now(UTC))
def test_enqueue_dequeue_persists_and_compacts(tmp_path: Path) -> None:
path = tmp_path / "items.jsonl"
with FileBackedQueueAdapter(config=_config(path, dedup_keys=("id",))) as queue:
queue.enqueue([{"id": 1}, {"id": 2}, {"id": 1}])
assert queue.size() == 2
assert path.is_file()
lines = path.read_text(encoding="utf-8").strip().splitlines()
assert len(lines) == 2
batch = queue.dequeue_batch(max_items=1)
assert batch == [{"id": 1}]
remaining = path.read_text(encoding="utf-8").strip().splitlines()
assert len(remaining) == 1
assert queue.size() == 1
def test_replay_on_connect(tmp_path: Path) -> None:
path = tmp_path / "items.jsonl"
config = _config(path, dedup_keys=("id",), age_key="created_at", max_age_hours=24)
now = datetime.now(UTC)
with FileBackedQueueAdapter(config=config) as queue:
queue.enqueue(
[
{"id": 1, "created_at": now - timedelta(hours=1)},
{"id": 2, "created_at": now - timedelta(minutes=10)},
]
)
with FileBackedQueueAdapter(config=config) as queue:
assert queue.size() == 2
assert [item["id"] for item in queue.dequeue_batch(max_items=10)] == [1, 2]
def test_replay_filters_by_max_age(tmp_path: Path) -> None:
path = tmp_path / "items.jsonl"
now = datetime.now(UTC)
path.write_text(
"\n".join(
[
f'{{"id": "old", "created_at": "{(now - timedelta(hours=30)).isoformat()}"}}',
f'{{"id": "new", "created_at": "{(now - timedelta(hours=1)).isoformat()}"}}',
]
)
+ "\n",
encoding="utf-8",
)
config = _config(path, dedup_keys=("id",), age_key="created_at", max_age_hours=24)
with FileBackedQueueAdapter(config=config) as queue:
assert queue.size() == 1
assert queue.dequeue_batch(max_items=10)[0]["id"] == "new"
def test_connect_skips_corrupt_and_invalid_lines(tmp_path: Path) -> None:
path = tmp_path / "items.jsonl"
path.write_text(
"\n".join(
[
"",
"not-json",
"[1, 2]",
'{"id": 1}',
'{"name": "missing-id"}',
]
)
+ "\n",
encoding="utf-8",
)
config = _config(path, dedup_keys=("id",))
with FileBackedQueueAdapter(config=config) as queue:
assert queue.size() == 1
assert queue.dequeue_batch(max_items=10) == [{"id": 1}]
def test_connect_skips_missing_or_invalid_age_on_replay(tmp_path: Path) -> None:
path = tmp_path / "items.jsonl"
now = datetime.now(UTC)
path.write_text(
"\n".join(
[
'{"id": 1}',
'{"id": 2, "created_at": "not-a-date"}',
f'{{"id": 3, "created_at": "{now.isoformat()}"}}',
]
)
+ "\n",
encoding="utf-8",
)
config = _config(path, dedup_keys=("id",), age_key="created_at")
with FileBackedQueueAdapter(config=config) as queue:
assert queue.size() == 1
assert queue.dequeue_batch(max_items=10)[0]["id"] == 3
def test_connect_is_idempotent(tmp_path: Path) -> None:
path = tmp_path / "items.jsonl"
queue = FileBackedQueueAdapter(config=_config(path))
queue.connect()
queue.enqueue([{"id": 1}])
queue.connect()
assert queue.size() == 1
queue.disconnect()
assert not queue.is_connected()
def test_evict_compacts_file(tmp_path: Path) -> None:
path = tmp_path / "items.jsonl"
config = _config(path, dedup_keys=("id",), age_key="created_at", max_age_hours=24)
now = datetime.now(UTC)
with FileBackedQueueAdapter(config=config) as queue:
queue.enqueue(
[
{"id": 1, "created_at": now - timedelta(hours=2)},
{"id": 2, "created_at": now - timedelta(minutes=5)},
]
)
removed = queue.evict_older_than(now - timedelta(hours=1))
assert removed == 1
assert queue.size() == 1
lines = [line for line in path.read_text(encoding="utf-8").splitlines() if line]
assert len(lines) == 1
def test_auto_evict_on_enqueue(tmp_path: Path) -> None:
path = tmp_path / "items.jsonl"
config = _config(path, dedup_keys=("id",), age_key="created_at", max_age_hours=1)
now = datetime.now(UTC)
with FileBackedQueueAdapter(config=config) as queue:
queue.enqueue([{"id": 1, "created_at": now - timedelta(hours=2)}])
# Item is enqueued then immediately age-evicted.
assert queue.size() == 0
def test_from_env_when_config_omitted(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
path = tmp_path / "from-env.jsonl"
monkeypatch.setenv("FILE_QUEUE_PATH", str(path))
with FileBackedQueueAdapter() as queue:
queue.enqueue([{"id": 1}])
assert queue.size() == 1
def test_injected_memory(tmp_path: Path) -> None:
path = tmp_path / "items.jsonl"
memory = MemoryQueueAdapter(dedup_keys=("id",))
config = _config(path, dedup_keys=("id",))
with FileBackedQueueAdapter(config=config, memory=memory) as queue:
queue.enqueue([{"id": 1}])
assert memory.size() == 1
def test_empty_enqueue_and_dequeue_skip_compact(tmp_path: Path) -> None:
path = tmp_path / "items.jsonl"
with FileBackedQueueAdapter(config=_config(path)) as queue:
queue.enqueue([])
assert queue.dequeue_batch(max_items=10) == []
assert queue.evict_older_than(datetime.now(UTC)) == 0
assert not path.exists() or path.read_text(encoding="utf-8") == ""
def test_reconnect_replays_after_disconnect(tmp_path: Path) -> None:
path = tmp_path / "items.jsonl"
queue = FileBackedQueueAdapter(config=_config(path, dedup_keys=("id",)))
queue.connect()
queue.enqueue([{"id": 1}])
queue.disconnect()
queue.connect()
assert queue.size() == 1
queue.disconnect()
def test_json_default_rejects_unsupported() -> None:
with pytest.raises(TypeError):
FileBackedQueueAdapter._json_default(object())
def test_disconnect_ignores_fsync_oserror(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
path = tmp_path / "items.jsonl"
queue = FileBackedQueueAdapter(config=_config(path))
queue.connect()
def boom(_fd: int) -> None:
raise OSError("fsync failed")
monkeypatch.setattr(
"python_repositories.adapters.file_backed_queue_adapter.os.fsync",
boom,
)
queue.disconnect()
assert not queue.is_connected()
def test_append_items_requires_open_handle(tmp_path: Path) -> None:
path = tmp_path / "items.jsonl"
queue = FileBackedQueueAdapter(config=_config(path))
queue.connect()
queue._append_handle = None # noqa: SLF001
with pytest.raises(RuntimeError, match="append handle is not open"):
queue._append_items([{"id": 1}]) # noqa: SLF001
queue._connected = False # noqa: SLF001
+56
View File
@@ -0,0 +1,56 @@
"""Unit tests for FileQueueConfig."""
from __future__ import annotations
from pathlib import Path
import pytest
from python_repositories.config import FileQueueConfig
def test_defaults() -> None:
config = FileQueueConfig(path=Path("/tmp/queue.jsonl"))
assert config.path == Path("/tmp/queue.jsonl")
assert config.max_age_hours == 24
assert config.dedup_keys == ()
assert config.age_key is None
def test_from_env_loads_path(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setenv("FILE_QUEUE_PATH", "/data/items.jsonl")
monkeypatch.delenv("FILE_QUEUE_MAX_AGE_HOURS", raising=False)
config = FileQueueConfig.from_env(use_dotenv=False)
assert config.path == Path("/data/items.jsonl")
assert config.max_age_hours == 24
def test_from_env_loads_max_age_hours(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setenv("FILE_QUEUE_PATH", "/data/items.jsonl")
monkeypatch.setenv("FILE_QUEUE_MAX_AGE_HOURS", "48")
config = FileQueueConfig.from_env(use_dotenv=False)
assert config.max_age_hours == 48
def test_from_env_raises_when_path_missing(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.delenv("FILE_QUEUE_PATH", raising=False)
with pytest.raises(Exception):
FileQueueConfig.from_env(use_dotenv=False)
def test_from_env_respects_custom_env_var_names(
monkeypatch: pytest.MonkeyPatch,
) -> None:
monkeypatch.setenv("CUSTOM_QUEUE_PATH", "/custom/q.jsonl")
monkeypatch.setenv("CUSTOM_MAX_AGE", "12")
config = FileQueueConfig.from_env(
"CUSTOM_QUEUE_PATH",
max_age_hours_env_var_name="CUSTOM_MAX_AGE",
dedup_keys=("id",),
age_key="created_at",
use_dotenv=False,
)
assert config.path == Path("/custom/q.jsonl")
assert config.max_age_hours == 12
assert config.dedup_keys == ("id",)
assert config.age_key == "created_at"
@@ -1,13 +1,45 @@
"""Unit tests for JsonRepositoryInterface."""
from collections.abc import Iterator
from typing import Any
import pytest
from python_repositories.interfaces.json_repository_interface import (
JsonRepositoryInterface,
)
class InMemoryJsonRepo:
"""Plain class that satisfies JsonRepositoryInterface without inheritance."""
def get(self, key: str) -> dict[str, Any] | None:
return None
def set(self, key: str, data: dict[str, Any]) -> None:
pass
def delete(self, key: str) -> None:
pass
def list_keys(self, pattern: str) -> list[str]:
return []
def scan_keys(
self,
pattern: str,
*,
count: int | None = None,
) -> Iterator[str]:
del count
yield from self.list_keys(pattern)
def accepts_json_repo(repo: JsonRepositoryInterface) -> None:
"""Type-checking hook for JsonRepositoryInterface structural subtyping."""
repo.get("key")
def test_instantiation_fails_when_get_not_implemented() -> None:
"""Test that instantiation fails if get is not implemented."""
@@ -103,3 +135,10 @@ def test_scan_keys_defaults_to_list_keys() -> None:
repository = Complete()
assert list(repository.scan_keys("user")) == ["user-1", "user-2"]
def test_structural_subtyping() -> None:
"""Test that a plain class satisfies JsonRepositoryInterface structurally."""
repo: JsonRepositoryInterface = InMemoryJsonRepo()
accepts_json_repo(repo)
assert isinstance(repo, JsonRepositoryInterface)
+149
View File
@@ -0,0 +1,149 @@
"""Unit tests for MemoryQueueAdapter."""
from __future__ import annotations
from datetime import UTC, datetime, timedelta
import threading
import pytest
from python_repositories.adapters.memory_queue_adapter import MemoryQueueAdapter
from python_repositories.interfaces.queue_repository_interface import (
QueueRepositoryInterface,
)
def test_implements_interface() -> None:
assert issubclass(MemoryQueueAdapter, QueueRepositoryInterface)
def test_fifo_without_dedup() -> None:
queue = MemoryQueueAdapter()
queue.enqueue([{"id": 1}, {"id": 2}, {"id": 3}])
assert queue.size() == 3
assert queue.dequeue_batch(max_items=2) == [{"id": 1}, {"id": 2}]
assert queue.dequeue_batch(max_items=10) == [{"id": 3}]
assert queue.dequeue_batch() == []
assert queue.size() == 0
def test_empty_enqueue_is_noop() -> None:
queue = MemoryQueueAdapter()
queue.enqueue([])
assert queue.size() == 0
assert queue.enqueue_and_return_added([]) == []
def test_dedup_keeps_first() -> None:
queue = MemoryQueueAdapter(dedup_keys=("id",))
queue.enqueue([{"id": 1, "v": "a"}, {"id": 1, "v": "b"}, {"id": 2, "v": "c"}])
assert queue.size() == 2
assert queue.dequeue_batch(max_items=10) == [
{"id": 1, "v": "a"},
{"id": 2, "v": "c"},
]
def test_missing_dedup_key_raises() -> None:
queue = MemoryQueueAdapter(dedup_keys=("id",))
with pytest.raises(ValueError, match="missing dedup key"):
queue.enqueue([{"name": "x"}])
def test_missing_age_key_raises() -> None:
queue = MemoryQueueAdapter(age_key="created_at")
with pytest.raises(ValueError, match="missing age key"):
queue.enqueue([{"id": 1}])
def test_evict_older_than() -> None:
queue = MemoryQueueAdapter(age_key="created_at", dedup_keys=("id",))
now = datetime.now(UTC)
queue.enqueue(
[
{"id": 1, "created_at": now - timedelta(hours=2)},
{"id": 2, "created_at": now - timedelta(minutes=30)},
{"id": 3, "created_at": (now - timedelta(hours=3)).isoformat()},
]
)
removed = queue.evict_older_than(now - timedelta(hours=1))
assert removed == 2
assert queue.size() == 1
remaining = queue.dequeue_batch(max_items=10)
assert remaining[0]["id"] == 2
def test_evict_without_age_key_is_noop() -> None:
queue = MemoryQueueAdapter()
queue.enqueue([{"id": 1}])
assert queue.evict_older_than(datetime.now(UTC)) == 0
assert queue.size() == 1
def test_parse_age_rejects_unsupported_type() -> None:
with pytest.raises(ValueError, match="age value must be"):
MemoryQueueAdapter._parse_age(123)
def test_parse_age_aware_datetime() -> None:
from datetime import timezone
eastern = timezone(timedelta(hours=-5))
aware = datetime(2024, 1, 1, 12, 0, 0, tzinfo=eastern)
assert MemoryQueueAdapter._parse_age(aware) == datetime(
2024, 1, 1, 17, 0, 0, tzinfo=UTC
)
def test_parse_age_naive_datetime() -> None:
naive = datetime(2024, 1, 1, 12, 0, 0)
assert MemoryQueueAdapter._parse_age(naive) == datetime(
2024, 1, 1, 12, 0, 0, tzinfo=UTC
)
def test_parse_age_zulu_string() -> None:
assert MemoryQueueAdapter._parse_age("2024-01-01T00:00:00Z") == datetime(
2024, 1, 1, 0, 0, 0, tzinfo=UTC
)
def test_parse_age_naive_string() -> None:
assert MemoryQueueAdapter._parse_age("2024-01-01T00:00:00") == datetime(
2024, 1, 1, 0, 0, 0, tzinfo=UTC
)
def test_dequeue_rejects_negative_max_items() -> None:
queue = MemoryQueueAdapter()
with pytest.raises(ValueError, match="max_items"):
queue.dequeue_batch(max_items=-1)
def test_clear_and_snapshot() -> None:
queue = MemoryQueueAdapter()
queue.enqueue([{"id": 1}, {"id": 2}])
assert queue.snapshot() == [{"id": 1}, {"id": 2}]
queue.clear()
assert queue.size() == 0
assert queue.snapshot() == []
def test_thread_safety_smoke() -> None:
queue = MemoryQueueAdapter(dedup_keys=("id",))
errors: list[BaseException] = []
def worker(start: int) -> None:
try:
for i in range(start, start + 50):
queue.enqueue([{"id": i}])
except BaseException as exc: # noqa: BLE001
errors.append(exc)
threads = [threading.Thread(target=worker, args=(i * 50,)) for i in range(4)]
for thread in threads:
thread.start()
for thread in threads:
thread.join()
assert errors == []
assert queue.size() == 200
+1 -1
View File
@@ -5,8 +5,8 @@ from __future__ import annotations
from dataclasses import replace
from unittest.mock import MagicMock
import pytest
from minio import Minio, S3Error
import pytest
from urllib3.response import BaseHTTPResponse
from python_repositories.adapters.minio_adapter import MinioAdapter
@@ -3,11 +3,38 @@
from io import BytesIO
import pytest
from python_repositories.interfaces.object_repository_interface import (
ObjectRepositoryInterface,
)
class InMemoryObjectRepo:
"""Plain class that satisfies ObjectRepositoryInterface without inheritance."""
def get(self, object_name: str) -> BytesIO | None:
return None
def put(
self,
object_name: str,
data: BytesIO,
content_type: str = "application/octet-stream",
) -> None:
pass
def delete(self, object_name: str) -> None:
pass
def list_objects(self, prefix: str = "") -> list[str]:
return []
def accepts_object_repo(repo: ObjectRepositoryInterface) -> None:
"""Type-checking hook for ObjectRepositoryInterface structural subtyping."""
repo.get("object")
def test_instantiation_fails_when_get_not_implemented() -> None:
"""Test that instantiation fails if get is not implemented."""
@@ -97,3 +124,10 @@ def test_instantiation_fails_when_list_objects_not_implemented() -> None:
with pytest.raises(TypeError):
_ = Incomplete() # type: ignore
def test_structural_subtyping() -> None:
"""Test that a plain class satisfies ObjectRepositoryInterface structurally."""
repo: ObjectRepositoryInterface = InMemoryObjectRepo()
accepts_object_repo(repo)
assert isinstance(repo, ObjectRepositoryInterface)
+237
View File
@@ -0,0 +1,237 @@
"""Unit tests for PostgresAdapter instantiation and injection."""
from __future__ import annotations
from unittest.mock import MagicMock
import psycopg
import pytest
from python_repositories.adapters.postgres_adapter import PostgresAdapter
from python_repositories.interfaces import TableRepositoryInterface
from tests.conftest import TEST_POSTGRES_CONFIG
def _mock_cursor() -> MagicMock:
mock_cursor = MagicMock()
mock_cursor.__enter__ = MagicMock(return_value=mock_cursor)
mock_cursor.__exit__ = MagicMock(return_value=False)
return mock_cursor
def _mock_client(*, probe_raises: Exception | None = None) -> MagicMock:
mock_client = MagicMock(spec=psycopg.Connection)
mock_cursor = _mock_cursor()
if probe_raises is not None:
mock_cursor.execute.side_effect = probe_raises
mock_client.cursor.return_value = mock_cursor
return mock_client
def test_should_adhere_to_interface() -> None:
assert issubclass(PostgresAdapter, TableRepositoryInterface)
_ = PostgresAdapter(config=TEST_POSTGRES_CONFIG)
def test_should_have_logger_when_instantiated() -> None:
adapter = PostgresAdapter(config=TEST_POSTGRES_CONFIG)
assert hasattr(adapter, "logger")
assert adapter.logger is not None
def test_should_not_be_connected_when_instantiated() -> None:
adapter = PostgresAdapter(config=TEST_POSTGRES_CONFIG)
assert adapter._client is None
assert not adapter.is_connected()
def test_constructs_with_injected_config_without_env(
monkeypatch: pytest.MonkeyPatch,
) -> None:
monkeypatch.delenv("POSTGRES_URI", raising=False)
monkeypatch.delenv("POSTGRES_TABLE", raising=False)
adapter = PostgresAdapter(config=TEST_POSTGRES_CONFIG)
assert adapter._config == TEST_POSTGRES_CONFIG
def test_constructs_with_injected_client_without_env(
monkeypatch: pytest.MonkeyPatch,
) -> None:
monkeypatch.delenv("POSTGRES_URI", raising=False)
monkeypatch.delenv("POSTGRES_TABLE", raising=False)
mock_client = _mock_client()
adapter = PostgresAdapter(config=TEST_POSTGRES_CONFIG, client=mock_client)
assert adapter._client is mock_client
assert adapter._client_injected is True
def test_raises_when_client_provided_without_config() -> None:
mock_client = MagicMock(spec=psycopg.Connection)
with pytest.raises(ValueError, match="config is required"):
PostgresAdapter(client=mock_client)
def test_disconnect_does_not_close_injected_client() -> None:
mock_client = _mock_client()
adapter = PostgresAdapter(config=TEST_POSTGRES_CONFIG, client=mock_client)
adapter.disconnect()
mock_client.close.assert_not_called()
assert adapter._client is None
class CustomEnvPostgresAdapter(PostgresAdapter):
uri_env_var_name = "CUSTOM_POSTGRES_URI"
table_env_var_name = "CUSTOM_POSTGRES_TABLE"
primary_key_env_var_name = "CUSTOM_POSTGRES_PRIMARY_KEY"
def test_subclass_custom_env_var_names(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setenv("CUSTOM_POSTGRES_URI", "postgresql://custom/mydb")
monkeypatch.setenv("CUSTOM_POSTGRES_TABLE", "items")
monkeypatch.setenv("CUSTOM_POSTGRES_PRIMARY_KEY", "item_id")
adapter = CustomEnvPostgresAdapter()
assert adapter._config.uri == "postgresql://custom/mydb"
assert adapter._config.table == "items"
assert adapter._config.primary_key == "item_id"
def test_connect_with_injected_client_succeeds_when_probe_ok() -> None:
mock_client = _mock_client()
adapter = PostgresAdapter(config=TEST_POSTGRES_CONFIG, client=mock_client)
adapter.connect()
assert mock_client.cursor.return_value.execute.call_count == 2
def test_connect_with_injected_client_raises_on_probe_failure() -> None:
mock_client = _mock_client(probe_raises=psycopg.OperationalError("connection lost"))
adapter = PostgresAdapter(config=TEST_POSTGRES_CONFIG, client=mock_client)
with pytest.raises(ConnectionError, match="Could not connect to Postgres"):
adapter.connect()
def test_connect_with_injected_client_skips_validation_when_client_cleared() -> None:
mock_client = _mock_client()
adapter = PostgresAdapter(config=TEST_POSTGRES_CONFIG, client=mock_client)
adapter.disconnect()
adapter.connect()
mock_client.cursor.assert_not_called()
def test_connect_reconnects_when_existing_client_unhealthy(
monkeypatch: pytest.MonkeyPatch,
) -> None:
stale_client = _mock_client(
probe_raises=psycopg.OperationalError("connection lost")
)
new_client = _mock_client()
adapter = PostgresAdapter(config=TEST_POSTGRES_CONFIG)
adapter._client = stale_client
monkeypatch.setattr("psycopg.connect", lambda *args, **kwargs: new_client)
adapter.connect()
stale_client.close.assert_called_once()
assert adapter._client is new_client
def test_connect_skips_reconnect_when_already_connected(
monkeypatch: pytest.MonkeyPatch,
) -> None:
healthy_client = _mock_client()
adapter = PostgresAdapter(config=TEST_POSTGRES_CONFIG)
adapter._client = healthy_client
connect = MagicMock()
monkeypatch.setattr("psycopg.connect", connect)
adapter.connect()
healthy_client.close.assert_not_called()
connect.assert_not_called()
assert adapter._client is healthy_client
def test_fetch_one_raises_value_error_on_invalid_pk() -> None:
mock_client = _mock_client()
adapter = PostgresAdapter(config=TEST_POSTGRES_CONFIG, client=mock_client)
with pytest.raises(ValueError, match="Primary key must not be None"):
adapter.fetch_one(None)
def test_fetch_one_raises_connection_error_when_not_connected() -> None:
adapter = PostgresAdapter(config=TEST_POSTGRES_CONFIG)
with pytest.raises(ConnectionError):
adapter.fetch_one("some-id")
def test_upsert_raises_value_error_on_invalid_row() -> None:
mock_client = _mock_client()
adapter = PostgresAdapter(config=TEST_POSTGRES_CONFIG, client=mock_client)
with pytest.raises(ValueError, match="Row must be a dictionary"):
adapter.upsert("not-a-dict") # type: ignore[arg-type]
def test_upsert_raises_value_error_when_primary_key_missing() -> None:
mock_client = _mock_client()
adapter = PostgresAdapter(config=TEST_POSTGRES_CONFIG, client=mock_client)
with pytest.raises(ValueError, match="Row must include primary key column 'id'"):
adapter.upsert({"name": "Alice"})
def test_upsert_raises_connection_error_when_not_connected() -> None:
adapter = PostgresAdapter(config=TEST_POSTGRES_CONFIG)
with pytest.raises(ConnectionError):
adapter.upsert({"id": "alice", "name": "Alice"})
def test_fetch_all_raises_value_error_on_invalid_limit() -> None:
mock_client = _mock_client()
adapter = PostgresAdapter(config=TEST_POSTGRES_CONFIG, client=mock_client)
with pytest.raises(ValueError, match="Limit must be a non-negative integer"):
adapter.fetch_all(limit=-1)
def test_execute_raises_value_error_on_invalid_sql() -> None:
mock_client = _mock_client()
adapter = PostgresAdapter(config=TEST_POSTGRES_CONFIG, client=mock_client)
with pytest.raises(ValueError, match="SQL must be a non-empty string"):
adapter.execute("")
def test_execute_raises_value_error_on_invalid_params() -> None:
mock_client = _mock_client()
adapter = PostgresAdapter(config=TEST_POSTGRES_CONFIG, client=mock_client)
with pytest.raises(ValueError, match="Params must be a tuple"):
adapter.execute("SELECT 1", []) # type: ignore[arg-type]
def test_upsert_with_primary_key_only_uses_do_nothing() -> None:
mock_client = _mock_client()
adapter = PostgresAdapter(config=TEST_POSTGRES_CONFIG, client=mock_client)
adapter.upsert({"id": "pk-only"})
mock_cursor = mock_client.cursor.return_value.__enter__.return_value
insert_call = mock_cursor.execute.call_args_list[-1]
assert insert_call[0][1] == ("pk-only",)
def test_execute_raises_connection_error_when_not_connected() -> None:
adapter = PostgresAdapter(config=TEST_POSTGRES_CONFIG)
with pytest.raises(ConnectionError):
adapter.execute("SELECT 1")
+69
View File
@@ -0,0 +1,69 @@
"""Unit tests for PostgresConfig."""
from __future__ import annotations
import pytest
from python_repositories.config import PostgresConfig
def _set_required_postgres_env(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setenv("POSTGRES_URI", "postgresql://localhost/mydb")
monkeypatch.setenv("POSTGRES_TABLE", "users")
def test_from_env_loads_all_fields(monkeypatch: pytest.MonkeyPatch) -> None:
_set_required_postgres_env(monkeypatch)
config = PostgresConfig.from_env(use_dotenv=False)
assert config.uri == "postgresql://localhost/mydb"
assert config.table == "users"
assert config.primary_key == "id"
def test_from_env_reads_primary_key_from_env(
monkeypatch: pytest.MonkeyPatch,
) -> None:
_set_required_postgres_env(monkeypatch)
monkeypatch.setenv("POSTGRES_PRIMARY_KEY", "user_id")
config = PostgresConfig.from_env(use_dotenv=False)
assert config.primary_key == "user_id"
def test_from_env_defaults_primary_key_to_id_when_unset(
monkeypatch: pytest.MonkeyPatch,
) -> None:
_set_required_postgres_env(monkeypatch)
monkeypatch.delenv("POSTGRES_PRIMARY_KEY", raising=False)
config = PostgresConfig.from_env(use_dotenv=False)
assert config.primary_key == "id"
def test_from_env_raises_when_uri_missing(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.delenv("POSTGRES_URI", raising=False)
monkeypatch.setenv("POSTGRES_TABLE", "users")
with pytest.raises(Exception):
PostgresConfig.from_env(use_dotenv=False)
def test_from_env_raises_when_table_missing(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setenv("POSTGRES_URI", "postgresql://localhost/mydb")
monkeypatch.delenv("POSTGRES_TABLE", raising=False)
with pytest.raises(Exception):
PostgresConfig.from_env(use_dotenv=False)
def test_from_env_respects_custom_env_var_names(
monkeypatch: pytest.MonkeyPatch,
) -> None:
monkeypatch.setenv("CUSTOM_POSTGRES_URI", "postgresql://custom/mydb")
monkeypatch.setenv("CUSTOM_POSTGRES_TABLE", "items")
monkeypatch.setenv("CUSTOM_POSTGRES_PRIMARY_KEY", "item_id")
config = PostgresConfig.from_env(
"CUSTOM_POSTGRES_URI",
"CUSTOM_POSTGRES_TABLE",
"CUSTOM_POSTGRES_PRIMARY_KEY",
use_dotenv=False,
)
assert config.uri == "postgresql://custom/mydb"
assert config.table == "items"
assert config.primary_key == "item_id"
@@ -0,0 +1,100 @@
"""Unit tests for QueueRepositoryInterface."""
from datetime import UTC, datetime
from typing import Any
import pytest
from python_repositories.interfaces.queue_repository_interface import (
QueueRepositoryInterface,
)
class InMemoryQueueRepo:
"""Plain class that satisfies QueueRepositoryInterface without inheritance."""
def enqueue(self, items: list[dict[str, Any]]) -> None:
pass
def dequeue_batch(self, *, max_items: int = 1000) -> list[dict[str, Any]]:
del max_items
return []
def size(self) -> int:
return 0
def evict_older_than(self, cutoff: datetime) -> int:
del cutoff
return 0
def accepts_queue_repo(repo: QueueRepositoryInterface) -> None:
"""Type-checking hook for QueueRepositoryInterface structural subtyping."""
repo.size()
def test_instantiation_fails_when_enqueue_not_implemented() -> None:
class Incomplete(QueueRepositoryInterface):
def dequeue_batch(self, *, max_items: int = 1000) -> list[dict[str, Any]]:
return []
def size(self) -> int:
return 0
def evict_older_than(self, cutoff: datetime) -> int:
return 0
with pytest.raises(TypeError):
_ = Incomplete() # type: ignore
def test_instantiation_fails_when_dequeue_batch_not_implemented() -> None:
class Incomplete(QueueRepositoryInterface):
def enqueue(self, items: list[dict[str, Any]]) -> None:
pass
def size(self) -> int:
return 0
def evict_older_than(self, cutoff: datetime) -> int:
return 0
with pytest.raises(TypeError):
_ = Incomplete() # type: ignore
def test_instantiation_fails_when_size_not_implemented() -> None:
class Incomplete(QueueRepositoryInterface):
def enqueue(self, items: list[dict[str, Any]]) -> None:
pass
def dequeue_batch(self, *, max_items: int = 1000) -> list[dict[str, Any]]:
return []
def evict_older_than(self, cutoff: datetime) -> int:
return 0
with pytest.raises(TypeError):
_ = Incomplete() # type: ignore
def test_instantiation_fails_when_evict_older_than_not_implemented() -> None:
class Incomplete(QueueRepositoryInterface):
def enqueue(self, items: list[dict[str, Any]]) -> None:
pass
def dequeue_batch(self, *, max_items: int = 1000) -> list[dict[str, Any]]:
return []
def size(self) -> int:
return 0
with pytest.raises(TypeError):
_ = Incomplete() # type: ignore
def test_structural_subtyping() -> None:
repo: QueueRepositoryInterface = InMemoryQueueRepo()
accepts_queue_repo(repo)
assert isinstance(repo, QueueRepositoryInterface)
assert repo.evict_older_than(datetime.now(UTC)) == 0
@@ -0,0 +1,162 @@
"""Unit tests for TableRepositoryInterface."""
from typing import Any
import pytest
from python_repositories.interfaces.table_repository_interface import (
TableRepositoryInterface,
)
class InMemoryTableRepo:
"""Plain class that satisfies TableRepositoryInterface without inheritance."""
def fetch_one(self, pk: Any) -> dict[str, Any] | None:
return None
def fetch_all(self, *, limit: int | None = None) -> list[dict[str, Any]]:
del limit
return []
def upsert(self, row: dict[str, Any]) -> None:
pass
def delete(self, pk: Any) -> None:
pass
def execute(
self,
sql: str,
params: tuple[Any, ...] = (),
) -> list[dict[str, Any]]:
del sql, params
return []
def accepts_table_repo(repo: TableRepositoryInterface) -> None:
"""Type-checking hook for TableRepositoryInterface structural subtyping."""
repo.fetch_one("pk")
def test_instantiation_fails_when_fetch_one_not_implemented() -> None:
"""Test that instantiation fails if fetch_one is not implemented."""
class Incomplete(TableRepositoryInterface):
def fetch_all(self, *, limit: int | None = None) -> list[dict[str, Any]]:
return []
def upsert(self, row: dict[str, Any]) -> None:
pass
def delete(self, pk: Any) -> None:
pass
def execute(
self,
sql: str,
params: tuple[Any, ...] = (),
) -> list[dict[str, Any]]:
return []
with pytest.raises(TypeError):
_ = Incomplete() # type: ignore
def test_instantiation_fails_when_fetch_all_not_implemented() -> None:
"""Test that instantiation fails if fetch_all is not implemented."""
class Incomplete(TableRepositoryInterface):
def fetch_one(self, pk: Any) -> dict[str, Any] | None:
return None
def upsert(self, row: dict[str, Any]) -> None:
pass
def delete(self, pk: Any) -> None:
pass
def execute(
self,
sql: str,
params: tuple[Any, ...] = (),
) -> list[dict[str, Any]]:
return []
with pytest.raises(TypeError):
_ = Incomplete() # type: ignore
def test_instantiation_fails_when_upsert_not_implemented() -> None:
"""Test that instantiation fails if upsert is not implemented."""
class Incomplete(TableRepositoryInterface):
def fetch_one(self, pk: Any) -> dict[str, Any] | None:
return None
def fetch_all(self, *, limit: int | None = None) -> list[dict[str, Any]]:
return []
def delete(self, pk: Any) -> None:
pass
def execute(
self,
sql: str,
params: tuple[Any, ...] = (),
) -> list[dict[str, Any]]:
return []
with pytest.raises(TypeError):
_ = Incomplete() # type: ignore
def test_instantiation_fails_when_delete_not_implemented() -> None:
"""Test that instantiation fails if delete is not implemented."""
class Incomplete(TableRepositoryInterface):
def fetch_one(self, pk: Any) -> dict[str, Any] | None:
return None
def fetch_all(self, *, limit: int | None = None) -> list[dict[str, Any]]:
return []
def upsert(self, row: dict[str, Any]) -> None:
pass
def execute(
self,
sql: str,
params: tuple[Any, ...] = (),
) -> list[dict[str, Any]]:
return []
with pytest.raises(TypeError):
_ = Incomplete() # type: ignore
def test_instantiation_fails_when_execute_not_implemented() -> None:
"""Test that instantiation fails if execute is not implemented."""
class Incomplete(TableRepositoryInterface):
def fetch_one(self, pk: Any) -> dict[str, Any] | None:
return None
def fetch_all(self, *, limit: int | None = None) -> list[dict[str, Any]]:
return []
def upsert(self, row: dict[str, Any]) -> None:
pass
def delete(self, pk: Any) -> None:
pass
with pytest.raises(TypeError):
_ = Incomplete() # type: ignore
def test_structural_subtyping() -> None:
"""Test that a plain class satisfies TableRepositoryInterface structurally."""
repo: TableRepositoryInterface = InMemoryTableRepo()
accepts_table_repo(repo)
assert isinstance(repo, TableRepositoryInterface)
Generated
+500 -451
View File
File diff suppressed because it is too large Load Diff